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intrctuct  :c:. 


1. •  EACHRCUMJ 

STAh!vIR2  (System  fcr  Tactical  Assessment  cf 
N'ultisource  Messaees,  Tver.  Radar)  is  an  experiments  1 
computer  3  rosrram  created  as  part  cf  an  investigation 
into  new  methods  of  correlating  information  available  in 
a  naval  environment  in  response  to  tne  tactical  situation 
assessment  iTSA;  problem.  It  was  written  by  the  Tactical 
Ccmrana  and  Control  Division  of  the  Naval  Ocean  Systems 
Center,  San  Diego,  California.  STAN‘MIR2  has  served  as  a 
testbed  for  explorations  of  applications  of  rule-based 
artificial  intelligence  inference  systems,  witn  a  general 
and  flexible  approach  to  the  ranee  of  acceptable  inputs 
tc  the  system. 

In  practical  application,  information  in  a  oesic 
forrat  is  received  from  a  variety  of  sensors.  This 
information  is  processed  within  c  network  of 
interconnected  rules  or  conditional  statements  whica 
^ay  be  satisfied  by  this  information,  even 
responding  to  information  in  cases  involving  varying 
degrees  cf  uncertainty.  STAKMR2  is  capable  cf 
performing  deductions  based  or.  this  received  information 
tc  speed  the  decision  process  in  a  rapidly  changing 
environment . 
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Before  we  discuss  tne  specifics  of  STAMPERS  and  a 
method  of  testing  the  utility  of  this  decision  aid, 
background  information  will  be  presented  to  set  the  stage 
for  the  why,  where,  and  how  such  a  system  was  developed. 


5.  OBJECTIVES 


There  are  three  principal  objectives  for  this  thesis: 

1.  Identify  the  characteristics  of  an  artificial 

intelligence  prcgra~,  with  an  overview  of  the 

applications  to  a  specific  military  situation. 

2.  Propose  various  levels  of  evaluation  methodologies 
for'  STAIWIR2 . 

3.  Present  the  results  of  an  experiment  conducted 

with  STAMMR2  using  one  of  the  methodologies 

presented  . 
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II 


INTZI IIGaNCI 


A.  LIFINITIGN 

artificial  in  tel  1 igence  (aI;  is  defined  as  'the  study 
of  ideas  which  enable  computers  tc  ic  the  things  tnat  make 
people  seer  intelligent”  rRef.  lj  and  "the  science  of  making 
rachines  do  things  that  would  require  intelligence  if  done 
cy  men  ’  [Ref .  2  j  . 

The  intricate  processes  from  which  these  things 
have  evolved,  such  as  learning,  growth,  or  raturation 
are  bypassed  in  the  artificial  intelligence  program  tc 
create  a  facsimile  of  human  thought  and,  subsequently, 
iecis  icr.-o’eKing,  Rut  the  bypassing  is,  in  reality,  a  very 
careful  compilation  of  tne  elements  which  are,  or  seem  tc 
be,  parts  cf  the  human  thought  patterns  ar.d  strategies 
leading  to  decisions.  That  is,  human  thought  is  analyzed 
ar.d  dissected  into  discrete  elements  which  may  be 
recombined  artificially  to  mimic  human  intelligence. 

Alternatively,  differing  patterns  or  methods  are 

explored  to  uncover  new  routes  which  could,  in  fact,  lead  to 
similar  results. 

R.  DISCUSSION 

To  imagine  tne  thought  process  as  a  branched  tree, 
with  decisions  made  at  each  juncture,  cr  nede, 


li 


1  c 

a 

simplification 

which  does  not 

reflect 

the  complexity 

cf 

tn 

e  problem  cf 

describing 

nur  an 

thought.  The 

actual 

representa  tier. 

must 

consider 

many 

interconnected 

pat  hs , 

with  multi  tire 

ctioaal 

flow 

through 

those  paths. 

A  c  c 

r.  t  r  ol  mechanism 

must 

also 

be  considered  which 

carries  a  thought  throu6h  to  conclusion  and  mi nimi zes 
retraced  or  r.cn-prcducti ve  routes. 

a  goal  of  artificial  intelligence  inquiry  is  to 
duplicate  this  complex  structure  in  a  corputer  program.  A 
difficulty  lies  in.  the  multi-dimensional  nature  of  the 
thought  process  which  requires  equally  complex 
investigation.  Ey  examining  the  problem  from  the  viewpoint 
of  discrete  elements,  it  ray  be  possible  to  find  the 
combination  which  accurately  reflects  a  pattern  which  net 
only  duplicates  what  happens  in  the  case  of  a  specific 
problem,  but  also  a  general  case  which  may  be  applicable  to 
ether  prcolems. 

If  STAh:MlR2  can  duplicate  the  reasoning  process  of  a 
decision  maker,  with  a  data  base  and  memory  consistent  with 
the  situation  presented,  then  it  is  possible  to  use  this 
artificial  intelligence  program  as  an  "intelligent 
assistant"  in  the  tactical  situation  assessment  problem 
which  will  be  discussed  in  the  next  section. 

The  computer  is  the  "ideal  experimental  animal." 
Infinitely  patient  and  easy  to  cere  for,  the  computer 
requires  structured  instruction,  composed  of  discrete 
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elements,  whicn  it  cc^tines  as  prcgrammed .  The  comb  inat  lcr. 
and  recombination,  done  at  speeds  sufficient  to  expose 
unproductive  patns  rapidly,  can  provide  a  practical  test  of 
tne  precision  with  whicn  the  experinenter  defines  his 
thought  model  [Ref.  3]  .  Additions  or  deletions  which 
tne  exrerirenter  ray  hypothesize  as  crucial  (cr  useless)  to 
the  final  result  of  his  pattern  may  be  evaluated  with 
relative  ease.  Flaws  or  conceptual  mistakes  which  ray 
exist  are  likely  tc  be  detected  in  the  computer  because 
the  computer  generally  will  not  accept  ambiguity  cf 
instruction  and  may  simply  cease  execution  cf  the 
program.  However,  a  program  designed  tc  answer  a  certain 
sort  cf  question  does  not  guarantee  a  correct  answer 
[Ref.  4] . 

The  applicability  of  computers  and  programming  is  not 
tne  emphasis  cf  artificial  intelligence.  However,  the 
unambiguous  nature  of  the  program  is  mentioned  to  reinforce 
tne  "  transparency’’  cf  the  actual  machine  and  focus  cn  the 
precision  cf  understanding  human  thought. 

Using  artificial  intelligence  techniques,  research  has 
been  done  to  explore  vision,  problem  solving,  language,  and 
neurosis  [Ref.  5] .  Some  of  these  models  have  been 
successful  in  predicting  behavior  under  specified  conditions 


while  others 

have  been  at  least 

of  metaphorical  value 

in 

helping  tc 

understand  the 

possible 

mechanism 

cf 

intell igence . 

By  representing 

thought  and 

knowledge , 

in 
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ueneral,  in  a  useful  and  flexible  manner  it  is  also  possible 
tc  arrange  an  interaction  between  a  human  and  tne  machine  in 
a  task  oriented  environment. 

knowledge  is  defined  as  a  collection  of  facts,  the 
state  cf  knowing,  or  all  that  has  been  perceived  by  the 
human  mind  [Ref.  6] .  Knowledge  ray  be  in  human  or  machine 
-•eTfry  or  in  a  data  base.  It  is,  however,  also  in  the 
a ctual  r rocedu res  that  operate  on  or  by  reference  to  the 
data.  Tc  store  disjointed  facts  in  computer  memory  or  in 
a  data  base  from  which  the  labelled  information  units  may 
be  retrieved  is  net  the  goal  of  artificial 
intelligence.  The  lden t if icat  i on  cf  the  desired  fact  icr  a 
location  code  by  which  that  fact  may  be  retrieved)  is  a 
problem  of  database  management.  Tc  discern  which  fact  is 
reiuirei  is,  however,  the  end  of  a  series  of  steps.  In 
the  problem  solving  case,  the  solution  process  is  tae 
strategy  of  gaining  information  after  ”  a  forward  or 
backward  looking  reasoning  method'  [Ref  7]  . 

Analogies  to  other  human  processes  exist.  For  example, 
radar  provides  target  information  in  a  nanner  analogous 
tc  the  eye.  Data,  such  as  target  presence,  range,  and 
bearir.;  may  be  useful  to  the  operator  as  well  as  in  a  form 
which  can  be  compared  tc  characteristics  stcred  in  computer 
memory . 


The  specific  goal  of  STAWER2  is  to  practically  combine 
this  sensory  data  with  the  problem  solving  process  of 


data  receipt,  corparison  and  retrieval  usin*:  the  knowledge 
stored  ir.  the  artificial  intelliter.ee  prcsrraT. 


Ill 


TACTICAL  SITUATION  ASS  USSf-'"' T 


STAN^iRZ 


TACTICAL  S  ITUATIO  ASSTSSrL'iNT 


In  the  naval  context,  the  atility  cf  a  commander  tc 
accurately  assess  his  military  situation  require*  decisions 
about  his  environment.  While  all  decisions  need,  not  be  race 
cn  an  immediate  basis,  certain  cf  the*,  called  tactical, 


determine 

his 

immediate 

course  of  action. 

In  an  extreme 

example , 

the 

dec i s i cn 

tc 

launch  a  defensive 

weapon  is  based 

on  a  Judg 

me  r.t 

that  such 

an 

action  is  required 

based  on  the 

commander 

* 

s 

eval ua  tic 

•*  • 

A  statement  of 

the  tactical 

situation 

assessment 

'  TS 

A)  problem  includes 

consideration 

cf  the  inforration  available  tc  the  decision  raker. 

In  crier  tc  assess  a  military  pr obi  ex ,  the  evaluate? 
draws  upon  experience  and  training,  after  having  considered 
the  situation  presented  tc  aim.  The  assessment  cf  the 
tactical  situation  is,  however,  nighly  dependent  upon 
factors  asscciated  with  the  decision  *aker  himself. 

The  experience  upon  which  a  military  decision  maker 
draws  is  uniaue  tc  hi*  alone,  although  there  is 
certainly  a  similarity  of  career  patterns  and  situations 
common  to  many  individuals.  Unique  tc  that  individual  is 
the  ouallty  of  his  recollection,  his  emotional  cr  rental 
state,  the  perspective  of  the  individual,  and  the  length 
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cf  time  elapsed  since  tacse  events  frcm  which  the 
experience  was  gained. 

The  actual  aecision  raker  in  the  operations  center 
the  Ccmfcat  Information  Center  or  CIC)  cf  a  Navy  snip  is 
the  Tactical  Action  Officer  (TAC).  Luring  the  period  cf 

the  Vietnam  conflict,  tne  role  cf  the  Captain  cf  a  ship  as 
final  fighting  authority  was  altered  somewhat  from  the 
traditional  role.  The  Tactical  Action  Officer  concept  was 
proposed,  implemented  in  Navy  Peculations,  and  exercised  in 
ccmtat  [?.ef.  S]  .  The  concept  basicly  states  that  a  trusted 
sutord ire t e ,  trained  and  drilled  in  defensive  doctrine 
and  procedures,  acts  as  weapons  release  authority  in  the 
temporary  aosence  cf  the  Captain. 

The  specialized  formal  training  for  TAOs,  having  as 
prerequisites  proven  maturity  and  operational 

experience,  lasts  approxima tely  six  weeks.  The  student  is 
exposed  to  infornation  concerning  U.S.  weapons  systems 
end  those  of  potential  enemies.  After  this  is 
committed  to  memory,  'standard"  tactical  doctrine  is 
exercised  in  a  series  cf  increasingly  complex  scenarios. 
The  scenarios  are  p re  seated  in  sequences  allow ing 
questions  tc  be  answered  based  on  in  format i cn  normally 
available  to  the  TAC  (from  the  training,  experience, 
tactical  publ  icat  icr.s  , sensors  ,  and  intelligence  scurces). 


The  TAC  becomes  the  focus  of  the  Combat  Information 
Center.  All  serscr  reports  are  displayed  tc  him,  all 


r^ssc-'es  from  external  sources  are  F;*iver.  to  him  directly  or 
in  summarized  form.  He  can  stand  alone,  co  nis  cwr. 
ship,  or  be  electronically  linked  by  radio  to  TAC-s  or. 
other  snips,  each  =  c in^  tnrcv  %n  similar  procedures  and 
actions.  This  defines  a  sphere  of  knowledge  in  which  the 
TAC  operates,  ana  can  generally  be  delineated  by  the  ran^e 
cf  his  sensors.  A  correlation  function  is  required  when 
the  TAG  receives  'external"  reports  from  sensors  cr 
platforms  ether  than  his  own — in  effect  frem  a  sphere  cf 
iar-er  'radius. 

Assumptions  involved  in  reducing  many  TAG  activities 
to  computer  form  include:  a,  the  basic,  repetitive  mess 
of  information  can  be  reducea  to  a  computerized  database; 
b)  standardized  procedures  for  data  retrieval  and 
correlation  can  be  sirilarly  reduced;  c)the 

sensor/ intelligence  information  can  be  presented  ir.  c 
standard  ferrat;  and  d)  the  thought  process  by  which  the 
TAG  functions  ray  be  artificially  reproduced. 

r.  STACIES 

STAt'M°.2  is  a  revised  version  cf  3TAVMI?.,  a  System  fer 
Tactical  Assessment  of  P-  ul  t  i  source  f-  e  s  senes ,  Aver.  Hade  r .  It 
was  created  as  cart  cf  an  investigation  cf  correlation 
cethodolo^ies,  and  served  as  a  testbed  for  explorations  of 
applications  of  rule-based  inference  systems  to  the  tactical 
situation  assessment  !TSA;  problem.  ST  Ayr"  IP.  concentrated  cr. 
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the  specific  talk  of  rertaant  shit  discrimination  i'rcr  all 
contacts  reported  by  radar  ani  external  messages  ’?.ef .  • 

is  an  organizer  of  information.  It 
collects  in  formation  cy  receiving  •'"essares  ar.  i  ser.scr 
reports  (radar,  electronic  support  neasrres,  er.d  scr.er', 
and  organizes  this  raw  date  into  graphic  list  lays  and 
textual  ccr.rr.enta  ry  to  aid  ir.  tactical  situation 
assessreat.  Thrcu^n  the  use  of  specified  rules,  the 
system  combines  tais  information  to  draw  conclusions 
about  the  situation  in  the  vicinity  of  the  hore  ship.  loose 
combinations  ere  reflected  in  teth  the  STAH-‘7?2  display  e'-d 
correct  ary.  I he  syster  is  available  for  tne  exarination  of 
raw  data  as  well  as  information  about  w ny  and  hew  the 
conclusions  were  reached. 

STAivb7?2  deals  with  inforration  cn  a  real-tire  basis. 
As  a  message  is  received  from  a  sensor  or  infcrraticr. 
source  in  5TAhM?.2-readable  forrat,  it  tecores  part  of 
the  data  base  cn  which  the  rules  operate.  This  process  is 
•corp  licated  by  the  fact  that  tne  inforration  is  suspect,  and 
its  arrival  ray  net  be  in  chronological  order. 

Althou^a  S?*mF2  was  developed  in  response  to 


I.avy- 

sponsored  investigation  of 

the  TSA  problem,  the 

U.S . 

aZT  / 

nas  conducted  sinilar 

investigations  in 

the 

field 

cf  military  intellUyen 

ce  analysis  '?.ef . 

le] . 

rasea 

upon  a  detailed  stuuy 

cf  the  analysts" 

role , 

-etheis,  and  thought  processes  in  intelligence  production, 


it  may  be  pcs. 

:  V  ^  Q 

tc  streamline 

intelligence  analysis. 

Ir.  essence, 

tni  s 

u  rr cess 

us  ed 

by  tr.e  analyst  is  the 

'  ethodolo  .'7 

from 

wi.  i  ch 

the 

A I  pro  ?rar:  right  be 

developed  af 

ter 

a  d  e  s  c  r  i 

pt  i  ve 

rc^el  is  organized  ar.d 

exe  rc i sed  . 

The  manner 

i  n 

which  STA 

b  M£H2 

functions  may  best  be 

described  by  analogy  to  the  medical  diagnosis  system  called 
r-'YCIN  [Ref.  11].  f-YClN  is  a  problem  solving  system  by 
w  .1  i  c ~  a  physician,  responding  tc  aueries  frc"  the  program , 
receives  assistance  in  identifying  and  treating  bleed 
bacterial  infections.  The  basis  fer  the  expertise 
resident  in  f/  Y  C I  h  is  the  knowledge  obtained  frer 
interviewing  physicians  ar.d  experts  and  stored  i”  ever  222 
"productions".  Productions  are  nenor^  structures  or.  which 
the  program  operates  and  are  in  the  following  format. 

If  the  infection  type  is  primary-bacteremia , 

the  suspectea  entry  point  is  the  gastrointestinal  tract, 
ar.d  the  site  of  the  culture  is  one  of  the  sterile  sites. 

then  tnere  is  evidence  that  the  organism  is  bacteroides. 

Conceptually,  inference  rules  are  very  simple.  Ivery 
rule  attempts  to  retrieve  information  fror  remory  and,  if  it 
succeeds,  constructs  a  new  assertion  lr.  re-cry.  If  all  the 
conditions  are  matched,  tnen  the  rules  are  said  to  'fire". 
That  is,  all  the  actions  which  are  defined  in  the  rule  are 
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tner.  carried  out.  These  actions  ray  be  modifications  tc  t  ne 
data  base,  assigning  cf  confidences,  and  printing  cf 
formatted  descriptions. 

In  a  sinple  rule  interpreter,  the  system  maintains  a 
list  cf  rules,  each  with  conditions  wnicr.  pertain  to  a 
central  data  base,  whicn  is  constantly  being  upaated.  At 
regular  intervals  the  system  attempts  tc  fire  each  rule? 
that  is,  it  checks  to  see  if  the  rule  conditions  are 
currently  satisfied  in  the  data  base.  If  ere  cf  many 
conditions  is  not  satisfied,  a  rule  ray  fail  to  fire  and,  in 
this  case,  any  partial  work  dene  in  satisfying  the  rule  is 
lest.  In  addition,  each  rule  is  retried  at  the  ti'-e 
interval  witnout  regard  tc  the  nature  cf  tne  intervening 
changes  to  the  data  case,  duplica  ing  results  where  no 
changes  to  the  data  base  nave  occurred. 

Because  medical  diagnosis,  even  by  experts,  involves 
varying  degrees  of  uncertainty,  MYCIK  is  written  with  a 
so-called  certainty  factor  for  each  conclusion.  The 
physicians  upon  whose  information  the  productions  were 
established  were  also  asked  tc  provide  "strength 
values",  or  probabilities  cf  accuracy  and  confidence, 
for  their  assertions.  This  introduces  the  ability  to 
deal  with  uncertain  evidence  into  the  artificial 
intelligence  system. 

The  productions  used  to  reach  a  conclusion  are  part  cf 
the  conclusion  set.  ^'YCIN  answers  questions  about  how  end 
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why  c  fact  ics  established  cr  usee.  The  production  is 
recalled  i  by  rare  cr  number,  tc  snow  an  enumeration  of  the 
facts  presented  in  the  premise  set.  Ter.eral  facts  can  be 
questioned  without  resertinr  tc  tne  entire  diagnosis  because 
each  production  "stards  clone",  to  be  actively  examined  or 
questioned.  la  this  case  tne  steps  taper  to  arrive  at  that 
particular  fact  can  also  be  traced.  This  t ranine  may  act 
as  a  training  aid  for  the  user  ia  addition  to  presenting 
alternatives  wnicii  can  be  explored. 

Tie  critical  aspects  of  STAb'KIH^ 's  design  may  be  divided 
into  four  parts.  These  are  Te-ncry  (the  data  base!,  "ule 
interpretation,  explanation,  a  r.d  ere  phi  cs  . 

The  rules  of  STA^ERE  consist  of  conditions,  actions  and 
confidences,  with  conditions  and  actions  in  the  sare 
structure  as  assertions  with  allowances  mace  for  variables 
which  are  "bound"  tc  a  rule  by  a  binding  function  in  LISF, 
the  language  in  which  STAMKZR2  is  written  [Pef.  12],  These 
variables  which  are  applied  tc  the  rule  conditions  will  be 
evaluated  to  see  whether  tne  condition  succeeds  or  fails. 
If  it  fails,  a  "suspension"  is  created  which  corresponds  tc 
the  remainder  of  the  rule.  As  more  information  is  added  to 
the  data  base,  these  suspensions  which  can  use  the  new  data 
are  revived  and  continue  as  before  (either  corpietin^  or 
suspending  a^ain;.  A  suspension  contains  not  cr.ly  tne 
remaining  part  of  the  rule,  but  also  the  bindings 
established  by  already  satisfied  conditions,  if  any.  Tver. 
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wher  c  condition  succeeds,  there  ray  be  other  ways  for  it  tc 
be  satisfied,  so  a  suspension  is  left  behind. 

The  individual  rules  are  expressed  in  the  f c 11 cv ire 
f  orr . 


RCiIjl-NaNE 
(CONDITIONS 
( (  <ccndition  IP  ' 
1  <ccnditicn  2  y  ; 


{  <ccr.diticr.  n  P  , ) 
ACTIONS 

{ {  <actior.  1>  ) 

(  <action  2>  ) 


(  <actior.  my  } 

CON 5  <a  number  between  -1  ar.d  +1' 

PR  INFORM 

“This  is  a  sentence  describing  the  rule.”) 

STAbt'E?.2  deals  wit;.  information  or.  a  real-ti~e 
basis.  However,  this  process  is  complicated  by  tne  fact 
that  the  arrival  cf  reports  may  net  be  in  chronological 
order,  with  later  information  superseding  or  negating 
earlier  reports.  A  ’data  strear"  is  used  to  bind 
information  as  it  is  received  tc  tne  rule  condition  which 
i t  satisf  ies  . 

A  data  stream  ray  be  defined  as  a  sec uer.ee  rf 
values,  existing  over  time  in  a  remputetion.  if  a  program  is 
executed  in  a  conventional  language,  then  the  history  of 
successive  values  of  a  variable  forms  a  stream.  Thus,  in 


contrast  to  static  data  structures  such 


s  lists  ar.d  arrays 


where  cli  the  elements  exist  at  cne  tire.  streams  -re 
d.y  r.ar  i  a  nata  structures,  ana  are  addressable  objects  in 
LISF.  Tnly  the  new  aata  received  need  he  comparer  tc  the 

rules.  New  matches  are  added  to  the  end  of  the  stream,  r. 

temporary  "freezing"  cf  the  action  specified  hy  an  assertion 
occurs  until  after  the  .new  lata  is  read  onto  the  appropriate 
streams.  The  stream,  then,  allows  the  rapid  review  cf  stored 
information  without  the  penalty  cf  complete  review  cf 

potentially  irrelevant  data. 

Ir.  STAVM?2,  confidence  is  provides  in  the  rules  by  the 
creator  of  the  rule.  It  is  calculated  dynamically  on 
recues t  at  the  time  when  it  is  displayed  to  the  operator 
fcllcwir?  a  rule  match  and  is  net  stored.  Each  assertion 
will  have  as  its  confidence  the  combination  of  the 

confidences  of  the  rules  and  assertions  which  offer  evidence 
for  it.  A  confidence  calculation  is  highly  dependent  upon 
t ne  connective  t  n rough  whica  the  rules  and  assertions  are 
cc "’tired.  An  ANT  connective  will,  for  example,  display  the 
confidence  of  the  smallest  value  of  all  the  rules  combined. 
That  is,  the  conclusion  nay  never  be  stronger  taan  the 
weakest  piece  of  supporting  evidence. 

Turing  executicn  of  STAdfiSmS ,  the  user  will  see 
the  following  cycle  repeated  as  long  as  messages  end  reports 
are  received  into  the  message  input  file: 

1.  A  message  revert  is  received,  the  user  is  informei, 
ana  the  critical  information  in  the  message  is  printed 
for  reference. 
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2.  A  display  ,  shewing  the  area  situation  with  the  new 
i  m' c  r  re  t  ic  n  .  is  dram.  The  user  ray  rani  u  la  te  tcis 
i~a..e . 

7.  The  system  makes  sore  conr.entf  r  y  or.  the  conclusions  it 
can  r?c;a,  or:  the  basis  of  the  a e w  information. 

4.  If  ary  conclusions  were  reacned,  tne  user  is  -riven  the 
opportunity  tc  juery  the  system  about  the  contents  of 
its  rata  base. 

The  message  text  is  i  r.  the  basic  format  of  the  received 
ressa^e.  A  contact  rare  (if  available!,  is  displayed, 
followed  by  location  and  tire  of  message .  The  user  "cy  pause 
tc  examine  tnis  message  or  select  the  graphic  display  rode. 

Graphical  support  for  STAMM  i’H2  is  provided  by 
I5PTA,  a  software  package  developed  at  the  .'.aval  Co -an 
SysteT  Center  specifically  for  tactical  situation 

essessrert  [Ref.  12].  The  TSPLA  system  is  a  collection  of 
3CRTRAK  subroutines  that  allow  stcra-e,  retrieval,  and 
display  of  ship  ana  aircraft  tracks.  This  display,  with  the 
capability  tc  shew  -a  os  with  latitude  and  longitude,  -ay 
ce  r  -  : !  pr.  la  t  ed  by  the  user  tc  vary  the  scale  as  desired.  As 
seer  as  a  "essa^e  is  received  which  contains  a 
demonstrable  assertion  (such  as  a  cor.  tart  at  a  specific 
location,'  tne  display  will  appear  on  the  display  screen 
■if  a  display  is  available  ar.d  selected  ir.  the 
initialization  process).  Ixamples  of  basic  function  keys  ere 
are  listed  below. 

M  magnify  about  the  cursor  oy  a  factor  of  2 


reauce  about  the  cursor  by  a  factor  of 


c  e  r.  t  e  r 


center 

ins  view  at  cut  t.ne  cv. 

rccr 

positic^ 

set  typ 

e  site  to  smallest  or. 

t  he 

T Z ?: T P. C I X  -214 

return 

t c  command  mode 

detailed 

functions  exist,  with 

the 

full  listir.r  in 

»  n  C  rt 

Iscnnioai  recurrent  252.  Ins  user  ray  return  to  the 
cc-rand  level  to  temir.  tx.e  explanation  ana  query  procedure. 

The  explanation  s/ster  provides  two  prirar/ 
carat  i  1  it  ies  :  retrieval  cf  memory  and  inference  tracing, 
al thrush  there  would  appear  to  be  little  difficulty  in 
retrieving  memory  ccr.ter.ts  and  tracing  derivations,  the 
display  of  this  information  in  a  hur.ar.  readable  forrat  is  a 
major  consideration.  Tc  make  the  user  interface  as  natural 
as  possible,  the  explanation  system  provides  a  query 
laa^uafte  that  is  *  Zn6-iisa-like" .  This  lac^uase  is  an 
extrerely  limited  version  of  In^lisn,  which  includes  only 
certain  types  of  questions  and  rethoas  cf  phrasing  those 
M.uestioos.  However,  the  lar.i-uase  was  designed  sc  that, 
while  limited,  it  is  sutficient  to  cover  the  user's  needs 
without  making  its  shortcomings  apparent.  The  LIS?  fnetier 
r.SH’JSEF,  which  features  recognition  and  pr^mptinm ,  is 
combined  vitn  a  "p rettypri nter"  (to  add  words  like  ‘is"  and 
"of"  to  memory  contents)  to  simplify  the  user  interaction. 

A  surra ry  cf  the  queries  which  tne  STAivMR2  user  can 
ask,  with  .p-eneral  examples,  follows.  following  the 
initial  query  word,  only  these  inputs  shown  will  be  allowed 
by  tne  ASKUSIR  function.  The  connective  words  (in  lower 


case',  will  be  provided  by  the  "prettyprinter"  fRef.  14]. 
The  words  i r.  parentheses  may  be  inserted  by  the  user  as 
desired  . 

WH  AT  Forrat:  ’a  HAT  is  IT  HI  AN  A)  < RELATION '  (CF) 

<  ITEfO 

IxarplesVEiT  is  the  COURSE  of  SIGHTING! 

This  every  provides  ere  cf  several  ouesticn 
formats  for  asking  about  entries  in  the  data 
base. 

13  Format:  IS  < TEE )  <  ITEt*>  ,A  AN  TKE)<RSLATION> 

(OF)  <ITZM> 

Example : I S  RAZAR  the  SOURCE  cf  SIGHTING! 

This  question  form  allows  the  user  to  e s  1c 
about  specific  entries  in  the  data  base. 

TELL  re  about  Forrat:  TELL  m.e  about  <ANYTHING> 

Example:  TELL  me  about  SIGHTING! 

This  is  the  most  flexible  iuery,  and  allows 
the  user  to  ask  rules,  categories , 
relations,  or  specific  facts. 

WHERE  Format:  WHERE  is  <^C3JECT>  cr  WHERE  was 

<'CEJECT>  at  <;TIKE) 

Example:  WHERE  is  CONTACT? 

With  this  command  the  user  can  ask  about  the 
position  cf  a  platform,  merchant  lane,  cr 
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v 


EC'* 


WECSE 


*HC 


storm  at  the  present  tire,  or  in  the  future 
or  past. 

Fermat:  *HY  is  <ASSE?.TICN> 

Exarp le :  WHY  is  AZ234C 

'*ith  this  command  the  user  car.  find  cut  the 
primary  or  immediate  reasons  that  STAGERS 
used  to  conclude  any  assertion.  All  the 
rules  involved  in  the  decision  will  be 
displayed  . 

Format:  KCW  does  rule  <RlTLE>  apply  to 
<ASSZRTICN  > 

Example:  ROW  does  rule  IE-LANZ  apply  to  A334 
This  query  allows  the  user  to  find  out  what 
assertions  cr  facts  were  involved  in 
permitting  the  rule  to  help  conclude  the 
given  asserticn. 

Fermat:  WHOSE  SHELAT ION>  is  <ITF"> 

Example:  WHCSZ  TYPE  is  ME CHANT 

This  query  acts  as  a  partial  inverse  tc  the 

WHAT  «uery. 

Fermat:  '*EC  is  \THZ  AN  A'  <RZLATICN>  (CF) 
<ITIK> 

Example:  WHC  is  INS  ICE  IAN  FI 


ZS 


ZXPir.IMMA 
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This  section  presents  a  discussion  of  experimental 
methodology  which  may  be  applied  in  the  evaluation  of 
STANMR2.  Following  an  overview  of  general  issues,  and  a 
discussion  of  general  levels  c  ?  experimentation,  three 
specific  stages  of  experiment  are  presented.  Trcm  these,  one 
has  been  chosen  to  be  carried  cut  as  a  sample  experiment. 

The  design  of  an  experiment  is  based  on  the  question  to 
be  addressed.  An  experiment  tc  compare  systems  with 
different  speeds  of  message  receipt,  for  example,  will 
include  a  message  generator,  an  interval  for  testing,  and 
the  means  to  measure  differences  ir.  total  messages 
received.  The  criteria  for  "better"  may  te  the  higher  number 
of  messages  received  ir.  a  certain  time  or  the  shorter  tine 
needed  tc  receive  a  certain  nurber  of  messages.  These 
objective  measures  of  performance  are  established  prior 
to  the  experiment,  with,  sufficient  flexibility  to  present 
the  data  tc  satisfy  the  question. 

Ir.  more  corplex  evaluation,  the  questions  tc  be 
answered  may  be  difficult  to  define.  If  the  goal  of  the 
experiment  is  to  find  the  number  of  messages  received  and 
understood  during  a  definite  time  interval,  tne  criteria  for 
understanding  must  also  become  a  part  of  the  experimental 
procedure.  Interpretation  of  tne  results  of  such  an 


experiment  trier  tecc-mes  dependent  cr.  ar.  "accepts tie "  le,rel 
of  irderstardin.-,  <?r.  acceptable  number  of  messages,  or  a 
comb  icaticc  of  the  tvo.  Tais  ray  tnen  be  ar.  area  where  the 
objective  measures  give  wap  to  less  precise  but  eiuallp 
useful  subjective  evaluation. 

In  order  tc  test  the  suitability  of  an 
artificial  intelligence  system,  measures  of  performance 
-ray  be  difficult  tc  establish.  Tne  components  cf  the 
program  may  be  individually  examined  (the  structure, 
execution,  results),  or  the  overall  improvement”  in  a 
parameter  of  the  function  which  the  program  is  designed  to 
assist  may  be  tested.  However,  tc  show  that  tne  system 
world  respond  tc  a  euesticr.  cr  situation  as  uvickly  as 
a  human  decision  maker,  given  that  the  situation  were 
constrained  tc  wna t  the  program  and  the  r.uman  knew  and  tne 
information  flow  was  at  “normal"  speed,  may  prove 
nothing  mere  than  that  computers  are  faster  than  humans. 
Considering  the  difficulty  in  establishing 
meaningful  measures  of  effectiveness,  subjective  evaluation 
cf  the  capability  and  the  utility  cf  3?  AmmSr.2 .  by  tactically 
oriented  individuals,  ray  show  the  greatest  premise  at  an 
evaluation  technique. 

Ixperirer.tatior.  ray  be  performed  ir.  stages  ranging  from 
simple  static  debug /valiaa  tioc  experiments,  whirr, 
concentrate  on  tne  capabilities  cf  the  tecnnrlcgy  and 
require  few  resources,  to  more  extensive  technical 


eva luct  i c us  cf  STAt- V,F,E2  with 


/'a^ically  cha-  :?  ns  data 


base  in  ?  controlled  er.vi  rcr.rent  .  This  could  include 
cynamic  operational  evaluations  in  simulated  ccmrana  and 
control  er.vircnrer.ts  pitting  cppcsir.f:  forces  arainst  eacn 
other  under  various  scenarios  us  in*.  one  of  the  warfare 
enviroarent  sirulator  (>.'Z S)  war  *;ame  programs  [?.ef.  15]. 

rollowir.b  an  overview  of  three  possible  levels  of 
testing,  rore  detailed  procedures  for  these  levels  will  he 
presented . 

The  basic  level  of  experiment  consists  of  ir.putir.r  into 
a  data  base  various  static  threat  scenarios  (a  snapshot 
view  cf  tne  situation)  concerning  the  status  cf 


Blue  aaa  Cran£e 


task  forces. 


The  test  objective 


is  tc  evaluate  the  capabilities  cf  3  TAM-- TP.  2  tc  read  the  data 
case,  signal  the  existence  of  the  threats,  and  for m 
opinions  based  cn  specific  rules  selected  ty  the  test 
conductor)  concerning  other  information  in  the  form  of 
sensor  and  i •' * °1 1  ic  ence  reports. 


rore  realistic  command  and  control  environment  with 


dynamically  cnan.-ai n<  data  base.  A  detailed  scenario  is 
presented  via  #15  (or  its  variants),  presenting  a  vide 
rar.re  cf  inputs  tc  STAivr'.i;?.2  under  chan^in^  circumstances 
to  probe  the  limits  of  its  capabilities ,  e.r;.,  wnat  threat 
intensity  is  ’'euuired  tc  "overload"  STAVMR2 ,  cr  wr.at  is 
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the  length  oi'  .lire  i'ror  the  icstact  tne  report  is  rereived 
i;r. til  S7Abf-E32  signals  a  conclusion. 

In  the  rost  conpiex  level  of  testing,  7.ES  is  again  used 
wit:;  Grange  and  iiue  task  forces.  These  forces  are 
composed  as  closely  as  possible,  witnir.  the  capabilities 
cf  available  test  resources,  with  equipment  isenscrs, 
weapons,  etc)  and  platforms  tnat  represent  current  military 
inventories.  Experimental  empnasis  is  shifted  from  a 
technical  evaluation  tc  an  operational  evaluation  of  the 
value  cf  STAiv!dE?.L  tc  tne  decision  maker.  The  objective 
is  tc  determine  if  STAKMr.E  improves  the  ccn.mande r ' s  ability 
tc  make  rapid  and  accurate  decisions.  Operationally 
realistic  scenarios  are  the  desired  norm.  Each  scenario 
sucvld  be  replicated  with  different  players  with  sc~e 
trials  having  the  Blue  forces  operating  with  S TAKMP.2  and 
sene  without.  Gperaticnal  measures  cf  effectiveness, 
which  depend  on  the  scenarios  under  play,  are  selected  tc 
assess  the  operational  utility  of  STAi'  ME? .  In  addition,  the 
players  and  the  umpire  team  are  subjected  to  post-exercise 
interviews  and  questionnaires  tc  -e t  subjective  evaluations 
cf  the  worth  of  STAMPEP2. 

While  there  are  r.c  definitive  bcundaries  between  tne 
levels  cf  the  ferial  experiments,  the  rar.me  cf  ccnditicr.s 
must  be  designed  for  scenarios  that  art  octr  realistic  and 
informative.  The  actual  analysis  at  each  level  of  experiment 
-i-ust  take  into  account  the  limitations  c.f  the  scenario 
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.ere  re  tor  r?.ef.  16]  .  Che  three  proposed  levels  of  experiment 
will  new  oe  presented  in  detail. 

A.  STASE-CNI  EXPERIMENTS 

1 .  Objective 

Sta^e-cne  experiments  evaluate  the  capability  cf 
STAMMERS  to  identify,  in  a  static  environment,  the  tactical 
situation  that  confronts  an  afloat  task  force  and  tc 
iescribe  the  situation  semantically  and  r;rapfci  ca  1  ly  .  This 
tyre  cf  experirent  establishes  the  type  cf  threats  that 
STAMMERS  car.  evaluate  and  assesses  the  timeliness  of  the 
warnings.  The  experiments  tray  also  serve  to  provide  r  edbach 
through  which  STAMMERS  may  be  refined. 

2 .  Resources  Required 

a.  Technical 

The  STAMMZP2  pro&ran  and  associated  computer 
ardware  are  required  for  this  experiment.  At  present, 
STAMMZR2  is  resident  or.  the  TOPS  cZ  system  at  !.CSC,  San 
Tieec,  California. 

b.  Ei splay 

An  alphanumeric  display  system  will  be  required 
for  players  and  the  umpire  team.  In  addition,  a  graphical 
display  of  the  situation,  as  is  normally  renerated  fcy 
VIS  rust  be  available  tc  compare  it  with  that  presented  by 
STAMMR2. 
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c.  Scenario 

The  tactical  scenario  is  presented  as  a  series 

cf  operationally  realistic  aata  plates'  constructed  in 

advance  of  the  experiment .  Each  data  elate  is  a  data 

snapshot  describing  a  tactical  situation  at  a  given  tire. 

* 

The  plate  snculd  include  various  information  ( course,  speed, 
ser.sor  reports,  intelligence}  available  to  a  Tactical  Action 
Officer  to  describe  tne  situation  fully. 

d.  Personnel 

The  personnel  required  for  the  experiment 
include  observers  of  the  output  (threat  assessors),  a 
scenario  controller,  and  a  test  director. 

e.  Support  Program 

In  addition  tc  scenario  generating  requirements , 
software  tc  measure  end  record  the  ti-es  at  which  the 
messages  are  output  and  the  tines  designated  by  the 
observers  is  required.  This  program,  based  on  tne 
nr.FxiMT  TYPESCRIPT  feature,  records  ail  inputs  er.i 
outputs  to  selected  terminals. 

3 .  Cer eral  Context 

a.  Concept  and  .Need 

STAiv^LF.2  is  being  developed  to  interact  with  a 
trap nlcs  display  system  to  support  the  comrander's  decision 
processes.  In  order  tc  assess  the  value  cf  the  technology 
we  must  determine  its  capabilities  and  shortcomings.  ror 
S7A.f'vIR2  to  be  useful  tc  a  decision  ma>er  it  •'ist  be  able  tc 
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evaluate  threats  cy  a  comparison  and  collation  cf  ell-source 
reports  ar.d  display  the  situations  in  a  tirely  rer.:.er. 


b.  General  Situation  and  Scenarios 

A  suitable  number  cf  data  plates  snoulo  te 
■-‘evelcrei  tc  vary  the  tactical  sitraticr  significantly  ir 
doth  the  complexity  of  the  situations  ar.i  the  nature  ari 
number  of  the  platforms  involved.  A  data  plate  should  be 
selected  at  random,  its  number  recorded  ar.d  the 
ap:  rop  ria  tely  stored  data  input  to  tne  file  which  STAh'PI?.2 
will  read.  An  observer,  unaware  cf  the  contents  of  the  data 
plate.  Heritors  the  contents  of  the  display  terriaai, 
evaluates  the  ir  formation  received,  and  describes  the 


situations  as  he  sees  them. 

4 .  Ivaluaticn 

a  .  Data  collection 

The  following  data  should  be 
each  trial:  the  data  plate  iier.  tificatic 

situation  assessment  cf  tne  evaluator, 
situation  assessment  iiver.  by  STA^^IHS. 
b.  Analysis 

The  primary  areas  of  concern  for 
experirents  are:  a  comparison  cf  the  STA.M^”H2 
tne  human  assessments  of  the  threats,  end  a 
cf  the  tires  required  for  S7AbM?2 
different  types  of  situations. 


collected  fer 
n  number,  The 
and  the 

the  ste^e-cr.e 
output  with 
de  termi  r.ct  ion 
tc  process 
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Pest-test  cr.cl.ysis  /.ill  exarine  tr.e  -ate  sr.eet 
tc  ascertain  if  ST.ihT  ZR2  accurately  evaluated  or  described 
and  displayed  all  phase-one  situations.  "iis  is  ccrpared 
with  the  evaluators'  ability  tc  do  the  same  based  or  the 
same  date. 

e.  Anticipated  Results 

It  is  ar.  ticipated  that  SiAvMR£  will  s : y r. c  1  the 
existence  of  threats  with  very  little  tine  delay  ar.d  that 
the  situation  end  display  will  be  presented  accurately.  ror 
tr.cse  cases  where  complex  nultiple  situations  are  present 
sirultenecrsly ,  the  tests  may  indicate  an  overcrowding  of 
tne  aisplay  and  a  lack  cf  tire  tc  exercise  the  logic  trace 
capability  cf  STAMMER*. 

c .  Corrents  and  SreciaL  Instructions 

Each  experiTental  subject  will  =c  tnrrvgn  a  short 
STAhMER*  indoctri  r.ati  on  session  during  which  ne  is  .riven  c- 
sumrary  cf  tne  theory  and  irp  lerer.  tat  ion  cf  SlAb.MTR2,  a 
threat  briefing  tc  explain  the  scenario,  end  ar.  overview 
cf  what  is  expectec  of  hi”.  Care  should  be  taken  tc  include 
in  the  data  plates  as  many  situations  as  possible  that 
STAbMR2  has  been  designed  to  handle  in  realistic 
situations.  It  is  recommended  that  several  different 
operators  be  subjected  to  tne  complete  set  of  plates  and  be 
asked  tc  subjectively  evaluate  the  displayed  information  tc 


provide  information  whicr.  ray  be  useful  in  the 
follow-on  display  experiments. 


of 
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The  s  t  age-cr.  e  ex  peri  er.taticr  diccussei  ate/?  will 


need  to  be 

carried  out  in  the 

course  of 

a  full  a 

nd  ca  rc f u  1 

evaluation  of 

STAi-MRZ  as  a  tool 

in  tacti 

cal  asses 

c  p-o  -  1  t  V  c<  r 

the  purpose 

cf  this  thesis, 

howe ve r , 

only  the 

ste^e-twe 

experiment  (as  discussed  below)  was  actually  performed. 

r  .  S T AG S-T'a 0  EXPERIMENTS 

1 .  Objective 

The  objectives  cf  stase-tvo  experiments  are:  tc 
assess  the  technical  capability  of  STnMMIRZ  actions  on  a 
dynamically  changin?  data  base  arer.eratec  by  creatine 
threat  situations  through  the  exercise  of  VIS  (or  its 
variants);  tc  determine  tne  ' limits'  of  the  capability 
cf  STAMMS!;  and  to  obtain  subjective  opinions  about 
STAMMIR2. 

2 .  Pescurces  F.eanlred 
a  .  Techni cal 

The  S TAMM  I?. 2  pregram  ar.i  associated  computer 
hardware  are  required  for  this  experiment,  "he  Warfare 
Invironment  Simulator  (VIS)  program  is  available  on  the 
TIN  EX  system  at  NOSC. 
b.  Display 

An  alpnanurerio  display  system  i?  reuuired  *',cr 
players  and  the  umpire  team.  The  GINTSCC  system,  a  color 
mrephlc  display,  may  be  used  tc  presert  the  situation 
generated  by  «IS  . 


w hi ch  is 


c  .  ;  c  ena  :•  i  c 

An  o  pera  t  i  one  1  ly  r  e  e  i  i  s  t  i  c  s  ce  na.  ri  o  is 

generated  tc  test  STAdMP.2  in  a  tactical  s  i  t  l  a  t  i  c  r.  at  a 
-:i7er  tire.  The  scenario  should  include  various 
inforration  (course,  speed,  sense:  reports,  intelligence) 
available  tc  a  Tactical  Action  Officer  tc  describe  the 
situation  as  it  is  corn-ally  presented  by  a“S  ,  via 
visual  display  and  status  boards.  The  evaluator  will  be 
allowed  to  interact  with  units  under  his  ~or.trcl"  by 
crderir.2  course  and  speed  changes,  and  enabling  various 
sensors  available  to  him..  The  tect  director  ray  insert 
various  elements  cf  in telli^erce  as  desired. 

4.  Personnel 

The  personnel  required  for  the  experiment 
include  observers  cf  tne  output  'threat  assessors',  a 

scerario  controller,  and  a  test  director, 
e.  Support  Frccran 

In  addition  to  scenario  wenera  t  i  n_r  re  u  r.  i  renp  r  t  s  , 
software  to  measure  and  recorc  tne  tires  at  which  tne 
messages  are  output  and  the  times  designate!  cy  the 
observers  is  desirable.  This  urogram,  based  on  the 


AP.PAN 


TYPESCRIPT  feature,  records  all  inputs 


cutouts  tc  selected  terrinals. 


Ivclucticr.  questionnaire 


Folio '» in m  a  stud.,'  of  t  oe  measures  cf  operational 
effectiveness  which  ray  be  achieved  in  this  interactive 
experiment,  an  evaluation  questionnaire  should  ce 
constructed  to  establish  the  level  to  which  these  measures 
nave  teen  satisfied. 

3 .  Ter.eral  Contact 

a.  Concert  and  Need 

A  complete  technological  assessment  of  the 
capability  cf  STAPMR2  includes  ar.  evaluation  ir.  a  realistic 
environment  with  a  dynamically  chancing  data  base.  "he 
experiment  rust  subject  $7AnM?.2  tc  a  wide  cf  situations 
under  varying  levels  of  ser.scr  activity  in  an  effort  tc 

exercise  all  carabilities  cf  STAivM?.2  end  try  tc  .rcbe  the 
"limits"  of  those  capabil: ties .  These  limits  could  be  the 
size  of  data  base,  number  cf  reports,  or  freu.uency  cf 
reports  . 

s  b.  Tenerai  Situation  ar.i  Scenarios 

'aZS  scenarios  will  te  generated  tc  present  a 

variety  of  sensor  reports  and  levels  of  activity  by 
platforms.  An  evaluator  familiar  witr.  tactical  war  gaming 
will  observe  the  play  cf  the  pa~e  wr.ile  exercising  STAI-TIRZ 
tc  the  maxi run  extent  pcs  si  cle. 

Zacn  experimental  sutject  will  gc  tnrcumh  a 

short  STAh'hZhZ  indoctrination  session  during  which  he  will 
be  given  a  sumrary  cf  the  tnecry  and  i^ple~entat  icr.  cf 


STAfMP.2,  5  threat'  brief;  r.._.  to  explain  the  scenario,  ~nd 
e.n  overview  of  what  is  expected  cf  him. 

Turing  the  course  cf  the  actual  experimental 
run,  the  evaluator  may  be  assisted  by  the  VIS  operators  end 
test  director  in  the  mechanism  cf  input,  output,  ar.d  display 
to  rerove  extraneous  or  distracting  details  of  the  fare. 

4 .  Evaluation 

a.  fata  Collection 

The  following  data  should  be  collected,  for  each 
scenario:  the  times  at  which  the  date  case  Is 
updated,  the  situation  analysis  by  STAN*!??,  and  the 
situation  analysis  by  the  operator. 

These  data  will  be  augmented  by  the  situation 
assessments  made  by  toe  test  director  who,  having  become 
familiar  with  the  full  scenario,  can  assess  the  "ground 
truth"  by  having  Knowledge  cf  the  actions  cf  all  platforms 
throughout  the  scenario.  The  questionnaire  'iven  to 
each  operator  should  consist  of  questions  designed  to  assess 
tne  real  is-”  c f  the  s cenaric  and  the  utility  cf 
STA^IEE  as  objectively  as  possible. 

b.  Analysis 

Comparison  is  made  between  ♦he  situation 

» 

descriptions  by  STA!‘:.VI?.2 ,  the  evaluators,  ar.c  srcur.d  truth 
to  assess  the  general  accuracy  of  these  descriptions.  The 
questionnaire  results  ray  be  presented  in  tabular  form  and 
tne  objective  ccTments  discussed  in  the  conclusions. 
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r n  t i c i ..  a t  e  3  Results 


*nil  e  it  rr  ?./  be  possible  to  generate  saturation 
pcirts  for  STAKPI-.l,  -arifest  ir  tlr-e  "tlays  wnereio  the 
STA‘  M7R2  display  significantly  lags  behind  the  >.TS  scenario, 
the  serial  r  a  ture  cf  report  processing  would  see~  t  c 
preclude  this  possibility  f  providirg  the  iuery  frontier  cf 
ST  At' MR  2  is  not  extensively  usea. 

I .  C events  and  Special  Instructions 

The  scenario  tare  should  be  maintained  sc  that  a 
sriven  game  may  be  replicated  exactly  with  zero  variance  for 
different  operators  and  so  that  the  test  director  can 
thoroughly  deterrice  the  situations  as  they  exist  (or  ere 
about  to  exist).  Kevin-?  the  varies  cr  tare  vruid  allow  trials 
without  a  IS -t  rained  operators  to  input  the  instructions 
required  by  a  detailed  script.  In  addition,  a  r  observer 
could  monitor  the  Hue  forces  in  one  or  rare  cf  the 
scenarios  in  a  run  without  STAMdIRl  and  make  oc-pariscns 
with  the  exact  run  without  ST.-*.MIR2 .  Finally,  if  the  runs 
were  net  available  on  tape,  very  detailed  scripts  desoritina- 
all  of  the  actions  of  both  the  Hue  ard  Creruoe  forces  rimht 
be  written  for  each  of  the  scenarios  to  accomplish  the 
objectives  of  this  test. 
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C.  STAGE- THREE  EX  PERI '  EM  5 

1 .  Objectives 

The  sta«?e-three  experireat  aas  as  its  objectives  to 
provide  an  operational  evaluation  cf  the  military  utility  cf 
STA!^hER2  to  a  decision  raker  at  ar.  afloat  task  force  command 
and  control  center  in  a  simulated  environment  and  to 
evaluate  whether  STANhZRZ  improves  the  decision  raker's 
ability  to  rake  rapid  and  accurate  decisions.  As  in  the 
case  of  the  ster’e-one  experiment,  this  level  of 
experimentation  are  discussed  ne re  cut  were  net  performed  as 
part  cf  this  research. 

2 .  Resources  Ren ui red 
a .  Technical 

The  STALER!  and  V»ES  programs  and  associated 
computer  hardware  are  required  for  this  experirent. 
b  .  I  i  s  p  1  a  y 

Aa  alphanumeric  display  syster  is  required  for 
ea^h  tea-’’  cf  players  and  the  umpire  tea".  The  GZh'ISCC 
system,  a  color  graphic  display,  may  be  used  to  present  the 
situation  which  is  venerated  by  >'ES  . 
c .  Scenario 

An  cp  era  t  ic  r.ai  ly  realistic  scenario  is 
fC.nerated  to  test  STAMMRZ  in  a  controllable  tactical 
situation .  The  scenario  snri.ld  include  various 
inferratior.  ^course,  speed,  serscr  reports,  ir.t»-liiwe.roe ) 
available  to  a  Tactical  Action  Officer  tc  describe  the 
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d  . 

Personnel 

The  personnel  required 

for  the  exp^ri 

rent 

include  observer  tears  to  act  as  Flue  and  Grange 
iorces,  a  scenario  ccr.trciler,  ar.  venire  tear,  and  a  test 
director. 

e.  Support  Program 

In  addition  to  scenario  genera  tin.-;  re  qui rere n t s  , 
software  tc  reasure  and  record  t.'.e  tires  at  which  the 
ressages  are  output  an i  the  tires  desiccated  by  the 
observers  is  required.  This  p  rcgrar ,  based  on  the 
APPANET  TV?ESC?IFT  feature,  reccrds  all  inputs  end 
outputs  to  selected  terrinals. 

f.  Evaluation  questionnaire 

Following-  a  study  of  the  re  a  su  res  of  operational 
effectiveness  vhlcr.  ray  be  achieved  in  tr.is  interactive 
experirer.t,  an  evaluation  questionnaire  should  be 
constructed  tc  establish  the  level  tc  which  these  measures 
..ave  t'  a  satisfied. 
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3  .  Gere rcl  Context 

a.  Concept  arc  Need 

It  is  possible  that  the  value  cf  ST  A/  M?.2  as  a 
decision  aid  ray  only  he  assessed  in  an  operational 
scenario.  For  example,  it  -ay  enable  a  decision  -aker  to 
better  understand  the  threat  situation  which  faces  his 
forces  and  to  "'a he  -ore  accurate  and  timely  decisions. 
These  experiments  would  pit  two  orposir>:  decision  makers 
a?air.st  each  ether  in  a  simulated  command  and  control 
environment  and  seek  to  measure  the  value  of  JTAVMHS  by 
comparing  the  decision  Takers'  performance  operating  with 
and  without  STA^MI?.2. 

t.  General  Situation  ard  Scenarios 

The  operational  evaluation  of  ST  AH'' IF.  2  respires 
cnenat  icr.a  lly  realistic  scenarios  consisting  cf  Blue  and 
Crar.ce  task  forces,  which  w'uli  be  acted  out  with  each  tear 
piven  the  flexibility  cf  exercising  complete  control  over 
their  forces  as  loop  as  they  do  not  countermand  their 
ordered  missions.  For  each  scenario,  an  operational  measure 
cf  effectiveness  if-OI)  or  multiple  MOTs  should  be  selected 
and  used  to  assess  the  operational  utility  cf  STAFFS 2. 

Because  cf  the  free-*. ley  flexibility  -piven  tc 
the  Elue  and  Cranpe  force  corranders,  the  war  wares  and  the 
resulting  outcomes  may  vary  significantly  from  trial  to 
trial.  Consequently,  each  scenario  snculd  be  replicated  in 
such  a  way  as  tc  avoid  foreknowledge  by  a  playing  tea-.  This 
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car.  be  done  by  usinp  a  uurber  of  tears  '.based  on  the 
availability  cf  assets^,  with  multiple  scenarios.  by  varyin* 
the  use  ana  non-use  of  STAyyEr.2  by  a  tear,  (considering  that 
a  tea.T  is  expected  tc  play  mere  than  one  .^a^e) ,  a  comparison 
of  performance  cased  on  the  b'CEs  may  be  made. 

Earn  team  sneuli  pc  through  a  brief 
i  nice  t  ri  r.a  t  icr.  session  during  which  the  fcllcwir.p  points 
will  be  discussed:  the  theory  cud  irj lerentation  of 
ST- Abb1  EE 2 ,  a  ’threat"  briefing  tc  explain  the  scenario  and  an 
overview  of  the  mission,  and  an  overview  of  the  measures  of 
effectiveness. 

4  .  Ev^ lua  t  ion 

a.  Data  Collection 

The  fcLlcwinw  data  should  be  collected  for 
each  scenario:  tires  at  which  the  data  base  is 
updated,  situation  assessments  by  STAbTEP.2,  sit'.atlcr 
assessrents  by  the  tears,  actions  tapes  by  the  decision 
r.a>er ,  ar.J  data  required  tc  determ’  the  selected 
operational  measures  of  effectiveness. 

These  data  ray  be  au5rented  by  the  sitiation 
assessrents  rale  by  the  test  director  who,  having  be  cert 
fariliar  with  the  full  s'-er.eric,  can  assess  the  'proved 
truth"  by  bavin.-:  Knowledge  cf  the  actions  of  ell  platforms 
throughout  the  scenario.  Tne  questionnaire  riven  tc 
each  operator  should  consist  of  questions  J e s i c n e c  tc  assess 


the  realise  of  the  scenario  end  the  utility  of 
ST.-h- f- 1=2  as  objectively  as  possible, 
b.  Analysis 

Data  summaries  ray  be  r.ade  for  eaj.n  scenario  end 
tabular  displays  should  be  ^ade  cf  tne  KIs. 


A  correlation  analysis  should  be  rede  with  the 
tire  sesuer.ce  cf  actions  taker,  by  the  decision  raker 
based  on  the  situation  assessments  made  by  STAb'M?2  to  try 
to  determine  hew  much  the  rc^Tan^ers  utilize 
STA.'MR2  end  what  information  is  rest  useful  to  thtm  r. 
comparison  cf  the  ?are  outcomes,  as  reflected  by  the 
bOIs,  nay  be  made  using,-  analysis  of  variance  to  see  if 
there  is  any  significant  difference  due  to  STA:-b;FR2. 

A  comparison  may  be  made  of  the  rant  outcomes 
and  the  subjective  appraisals  cf  STAf-.yFE2  to  see  if  there 
was  a  relationship  between  how  well  the  forces  performed  and 
hew  much  they  liked  STAVMK2. 

c.  Anticipated  Results 

It  is  anticipated  t.nat  there  could  be  a  lar=re 
variance  in  the  overall  end-  cf-mame  operational  measures  cf 
effectiveness.  The  possibility  exists  that  the  ^CFs  will  be 
sufficiently  ambU-ucus  that  ar.y  signal  due  tc  SmAi"ivi'R2  -a y 
not  be  discernible.  Thus,  the  subjective  evaluations  of  the 
teams  ray  provide  trie  only  discrimination  in  the  test,  with 
no  iuentitative  assessment  of  the  value  of  5T*f->TE2 
possible. 
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Zee  correlation  analysis  of  the  decision  maker's 
actions  and  the  bTAt-’ME2  assessment  should  reveal  the  type 
cf  i  r.  format  i  or.  that  is  Test  useful  tc  c  decision  maker  and 
new  rush  he  may  £jrow  to  rely  cn  STAMZR2.  Overall,  these 
experiments  should  yield  important  information  a  tout  the 
acceptability  of  STA‘-'M?.2  to  the  decision  maker  and  its 
utility  to  him. 

i .  Ccmnents  and  Special  Instructions 

The  usefulness  cf  STAbMRA  tc  a  decisicn  maker  may 
strongly  depend  on  the  manner  in  which  STAHhIR2  outputs  are 
displayed  tc  the  Blue  forces.  This  m i h t  he  especially  true 
in  hish  density  situations  with  a  backlog  of  unprocessed 
ser.scr  reports.  The  nature  cf  the  display  dedicated  tc 
STAf-N.ER2  should  be  considered  prior  to  sta5’e  three 
experiments.  The  use  of  a  terminal  ether  than  the  graphic 
display  terminal  for  auery  and  response  actions  allows  the 
picture  to  remain  undisturbed  until  refinerer.t  of  scale  or 
sequential  processing  is  requested.  The  Jispiay  syste-  is 
an  integral  part  of  STAPKFEZ  so  the  evaluation  should  ce 
tased  cn  the  most  flexible  display  available. 

A  tape  record  of  the  o’oits  should  be  made  of  the 
inputs  ana  outputs  of  eaoh  ^are  so  that  evaluation  personnel 
can  reproduce  a  .piver.  trial  and  conduct  a  review  cf  any 
point  to  assist  in  "what  if"  types  cf  analysis. 
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The  scenarios  used  should  be  creraticraliy 
realistic,  and  tr.e  starting  cor.iiticr.s  and  measures  of 
effectiveness  must  ce  selectee  to  exercise  the  full  ranye  of 
STAP-  K  IP.2  capabilities  ar.d  t  c  yield  ls  eful  inf  cr^at  i  cr  .  The 
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not  be 
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Ah  M I  P.2  . 

Instead  , 

tr.e 
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Orange 

task. 
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possible, 

v  i  t  h  i  n 

the 

constraints  cf  resources,  the  cc~pc s i ti cr  cf  forces 
currently  available  or  projected  as  desired  and  the  types  of 
rissiens  normally  undertaken  by  ea  oh. 

The  subjects  used  in  the  experiments  ray  be  very 
irportant.  For  purposes  cf  scientific  credibility,  the 
subjects  "-ust  nave  ccn"-ar.d  and  control  experience  at  hi.ph 
levels,  preferably  Navy  Captains  (C-cl  or  Admirals.  A 
potential  criticises  cf  experiments  which  attempt  tc 
determine  the  operational  utility  of  a  system  is  that  the 
’ operators'  aid  not  reflect  the  potential  user  population. 
This  would  seen  especially  true  when  the  evaluations  depend 
heavily  on  subjective  appraisals.  There  are  practical 
difficulties  in  obtaining  suer,  test  subjects.  However,  the 
issue  is  whether  tnese  tests  can  be  vievea  as  scientific 
experiments  cr  demonstrations  without  tr.e  operational 
realisr  engendered  by  their  presence. 
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.i.  INSCRIPTION 

A  sample  exnerimert  was  selected  from  tr.e 
preceding  discussion  tc  illustrate  both  the  ca pebi  1  i ty  of 
STaF- K?R2  and  and  t  ne  practical  application  cf  these  methods. 

The  stare-two  experimental  methodology  was  selected  for 
this  thesis.  Selection  criteria  included  the  expertise  cf 
the  test  subjects  available  and  the  availability  cf  the 
facilities  at  the  Naval  Postgraduate  School.  The  ttst 
subjects  selected  were  military  officers  with  background  ir. 
rc^rer.d  control  concepts  rained  through  formal  education 
and  a  wide  ranre  cf  military  experience.  The  Ccrrar.d, 
Control,  and  Communications  Laboratory  contains  extensive 
rrapnic  and  text  capability,  and  the  STA.N«VI?2  ar.d  >'~S 
software  are  available  at  the  Naval  Ccear.  Systems  Center 
UMCSC',  San  L ieyo ,  California. 

The  Warfare  Lr.vi  re  ore r  t  Simulator  (VIS,  war  ra^e 
procedure  involves  the  construction  of  a  computer  file  voice, 
contains  identification  cf  all  - es i red  players,  initial 
positions,  co'.:rses,  ar.i  speeds.  Platforms  may  te  selected 
f rc^  a  resident  data  base  vhic.n  assigns  senscr  and 
weapons  configurations  tc  the  units.  rc 11  ovine  mare 
initialization ,  the  u-its  may  te  -areuvered  cr  cc — and  *rc~ 
the  player  through  a  computer  terminal,  with  display 


of  exact  jeo«?rat.iic  location  of  >nc*n  units  available  or. 
selected  crap hie  disci  ay  reciters.  Interact!  cn  cf  tr.e 
t  nits  ray  be  ordered  by  trie  flayers  (as  in  attacks),  but 
sensor  recerts  are  deuender.t  c1"  detection  alcrritn^s  ir. 


V.ZS.  For  example,  detections 


surface  search 


radar  ayainst  another  surface  vessel  cc-urs  at  2?  nils 


minimum  separation 


er.  sor  reports  are  =,iven  to  the  cl  a  ye 


via  a  status  board  text  display  and  on  the  yecyraphic 
display  which  is  updated  every  .pare  rir.ute. 

This  exper  ire  n  t  was  r.ct  interactive  in  that  the 
operator  did  net  influence  the  series  cf  events  which 
were  bein'  generated  by  '.v'l S.  It  was,  however,  interactive 
ir.  the  sense  that  the  operator  add  tr.e  capability  to  select 
different  status  board  displays  and  change  ranye  scales 
as  dssirea.  The  fully  interactive  war  care  as  su*  res  ted  in 
the  third  level  cf  experiment  would  allow  operator 
influence  of  the  event  stream.  The  prepare-*  scenario 
used  in  this  experinent,  wnicn  included  initial 
positions,  initial  courses  end  speeds,  irb  orders  tc 
the  various  units  are  included  in  Appendix  I. 


The  scenari 


this  experiment,  venerated  ir 


*IS,  consisted  cf  three  U.5.  warships  ccnductiry  a 
surface  surveillance  of  a  rrerc.nant  lane  in  the  vicinity  cf 
the  Perth  Atlantic,  vita  neutral  ard  • potential ly !  hostile 
platforms  in  the  area.  The  warships,  assisted  by  patrol 
aircraft  ccndvctinr  similar  surveillance,  were  tc  locate, 


track,  ar.d  identify  ct:.e  r  units  in  the  area.  "he 
expe  r  irer.ta  1  subjest,  embarked’  on  t  ne  cruiser  Belknap,  nad 
STAbMP.Z  available  to  assist  ir.  that  mission.  The  STAt-PIPk 
startup  procedure  is  miven  in  Appendix  A. 

The  scenario  was  chosen  based  or.  the  re r me  of  situations 
available  and  the  relative  realism  of  a  nc  r.-comcat  ive 
confrontation .  The  scenario,  tactics,  s  e  n  a  r  r  c  ,  a'-!  sensor 

c.naracteris t ics  are  purely  hypothetical .  The  uata  base 
rApcendix  B]  and  rules  .Appendix  C]  are  assumptions  tailored 
to  this  scenario,  but  are  easily  translatable  to  those  which 
a  Tactical  .Action  Officer  would  reeemr.  ize  as  hav  ir.  m 
potential  ar pi i cations .  The  briefing  cf  the  subjects 
included  ar  explanation  cf  the  artificialities. 

A  difficulty  in  the  actual  conduct  of  the  experiment 
existed  due  tc  t.ne  nature  cf  the  reports  or.  whirr.  STA:- mEP.Z 
operates.  The  sensor  reports  received  by  an  operator  cf  *1 
are  both  visual  cues  or  a  mrapnics  display  'lines  cf  tearir.y 
cr  r.ew  platform  symbol omy ) ,  ana  in  a  table  of  ..detections 
l the  Electronic  Warfare  status  Beard,  the  Surface  Status 
Board,  or  the  Air  Status  Board).  The  form  o'  the  tabular 
report  is  formatted  as  to  its  numerical  secuer.ee, 
identification,  and  location  * i f  applicable;.  The  reports  as 
.generated  by  :VIS  are  net  readable  by  STAYMF.2.  Cae  example 
cf  the  disparity  is  that  STAb’bERL  recognizes  latitude  and 
lonmitude  in  decrees  and  tenths  rather  than  decrees  ar.d 
-irutes.  The  LISP  version  cf  jES,  L> ES ,  does  report  in  the 
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?  i.  i  rcpr  ia  te  ferret,  tut  the  difficulty  ir  usir..;  l'.;~S  ,  such 
as  complex  scenario  veneration  and  limited  output 
capability,  far  cu  twei.’h  this  advantage. 

The  solution  to  tie  format  disparity  lies  :  r.  the 
nature  of  ’vIS.  with  duplicate  platforms,  starting  at  exact 
locations,  given  precisely  the  same  order  cf  instructions 
i such  as  course  changes  cr  sensor  status',  sense r  reports 
are  also  duplicated.  That  is>.  the  detections  are  based  cn 
maximum  ranges  of  detection  which  are  unchanged  and 
vnich  are  combined  in  the  *ES  detection  algorithms  ir.  the 
same  way,  to  produce  duplicate  reports  at  the  same 
ga-"e  ti^e.  This  characteristic  was  exploited  by  running  a 
gars  to  a  suitable  point  ir.  time  using  the  ARPaMT 
typescript  feature  which  copies  ail  ter^ine* 
interactions  as  they  occur.  This  typescript  shows  the 
various  reports  received  during  a  game,  complete  with  the 
time  cf  receipt.  Synthesizing  ea^e  reports,  a  ''cmpiete 
file  of  reports  [Appendix  D  j  was  built,  in  the  format 
fre"  which  STAFMZRZ  could  read  its  input  data.  Therefore, 
STAP-f”E?.2  was  net  receiving  direct  output  from  the  sensors, 
tut  via  an  intermediate  process  net  part  cf  eituer  cr 
SIA^EF.2. 

STAf‘TE?.2,  using  the  TidTRCNIX  4Z14-1  terminal  for  both 
text  and  graphic  display,  ves,  in  effect,  a.  separate  entity 
running  concurrently  witn  the  game.  The  comparison  of  the 
subject's  perspective  of  the  situation  and  that  cf  STAA't'F^Z 
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is  still  valid  only  as  1  or.-'  as  s  ir  ul  ter.ecus  reporting  is 
carefully  ^aintaine-*  . 

The  artificiality  of  the  message  i n p u t  uonld  nave  biased 
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f e a  t ure  which  c 

meliorates  this 

..ro  bl  er 

• 

STANUIR 2  d  pec 
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process 
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report 

in 

the  file  until 
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u  s  e  r  j  u  i  t  s 
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query  rode  of 
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c  r. 

"hat  is, 

unti 

1  the 

quit 

command  is 

-q  i  v  e  r. 

there  is  nc 

processing  of  the  next  sequential  report.  Receuse  the 
experimental  subject  is  r.ct  expected  to  te  the  actual 
operator  of  the  ST.r.MM?2  hardware,  the  operator  car,  be 
instructed  not  tc  leave  tne  Query  mode  until  the  appropriate 
sane  time  (which  is  displayed  or.  The  input/output  terminal 
for  ’*ZS  ) . 

icllcwir.g  a  suitable  period  of  fame  time  where  the 
subject  is  encouraged  to  exercise  STAN VER2  a  questionnaire 
was  presented  to  gather  the  impressions  and  appraisal  of  the 
experiment  in  general  ana  STAmihZP.2  in  particular. 

£.  RESULTS  AN L  CONCLUSIONS 

The  questionnaire  supplied  to  test  subjects,  with  a 
summary  cf  results,  is  listed  telcw.  The  intent  cf  the 
questions  was  to  verify  the  "realism"  of  the  scenario  and 
tc  gather  a  subjective  evaluation  cf  STA^HZP.2.  The  scoring 
scale  of  1  to  10  was  arbitrarily  chosen,  with  10  the 
rost  positive  or  most  favorable  response.  The  test 
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subjects 


cons  is  ted 


of  eight  «a?y  and  Air 


rar.t-ir..t  ir. 
tr.e  Command 


^rede  from.  C-2  to  C-5  ,  who 
,  Centre!  ard  Cemm.ur.  ica tiers 


Force  officers  , 
are  members  of 
curriculum  at  the 


ravel  Postgraduate  Schccl,  bor.terey,  California.  All  have 
beer  exncsed  tc  decision  theory  ard  to  an  overview  of  the 
revel  tactical  environment.  The  Navy  re  r  t  i  ci  pa  r.  t  s  included 
three  C-4s  and  one  0-2.  All  hut  one  c*  the  havy  0-4S  had 
Tactical  Action  Officer  training. 


STAMPERS  QUESTIONNAIRE 


1.  Prior  to  the  scenario  presentation,  was  the  purpose  of 
£TAhhFH2  adequately  explained?' 

1224067  =  912 


responses  1122  1 

Average  6.125 

2.  a  s  the  scenario  representative  cf  your  previous 
exrerience  with  >ES  or  L>ES? 


Responses 
average  £.275 


it 


2.  'a as  the  scenario  realistic? 

1  2  2  4  5 
Responses  1 
Average  6.275 


6  9  1” 

1  2  2 


,ere  t::e  displays  consistent  wit::  the  sersc"  reports? 


Responses  2  2  2  1 

Average  r.375 

i.  Was  sufficient  tire  available  to  understanc  the  senso 


reports  ? 


responses 
Average  7.5 


7 

2 


12 


.  Were  the  STAMMR2  .graphics  clear? 

1  C  "*  A 


ie 


Responses  1  13  2  1 

Average  4.5 

7.  Was  the  STAMMS!  commentary  understandable? 


Responses 
Average  £.57; 


o 

1 


lr 


2.  Were  the  STAMM1S2  conclusions  consistent  with  you 
conclusions? 


ir 


Responses  1 

Average  7.25 

3.  "id  you  use  the  explanation  trace? 

12  3  4  5c 

Responses  1 

Average  ? .375 
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1 2.  If  the  exrlar.ati.cn  trace  was  used,  were  the  assertions 
consistent  with  the  ccnclusicrs? 

1234££r'S?l? 


Responses  2  3  1 

Average  7.375 

11.  >as  the  presence  of  STAhMRc  distracting? 


1 

£  3 

4  t  6 

7  E  9 

It 

Responses  1 

'Z 

£ 

l 

Average  4.75 

12.  i^ou  Id  an 

STATMR2  to  the 

assistant  who  would  filter  inforratior.  fror 
r i r. irurr  essential  level  be  useful? 

1 

2  3 

4  h  c 

7  c  9 

12 

responses 

1  1 

3 

t 

Average  5.5 

13.  Is  S TA/WVZ?2 

useful? 

1  2 

4  K  (= 

7  8  9 

17 

Responses 

1  1 

4  2 

Average  7.75 

The  results 

shew  an 

overall  trend 

towards  an 

average 

response  which 

could 

be  considered 

favorable , 

in  the 

vi-inity  of  £. 

The  exc 

eptions  previd 

e  a  clear  c 

o  n  t  r  a  s  t 

consistent  with  conditions  noticed  during  the  experirer.t. 

Question  5  had  the  lowest  average  ranging.  r"r.e 
difficulty  encountered  with  the  display  nay  be^T  he 
described  by  shoving  the  display  at  three  selected  care 
"inutes.  figure  1  is  the  last  display  without 

Figure  2 


bearing  lines,  which  occurs  at  rinute 


1 


is  the 


display  at  ~a~e  minute  14.  wr.icr.  includes  intelligence  data 
fM?.l  and  !'•’!?£),  surface  contact  reports,  !’•  bearin'  line? 
and  tne  location  of  friendly  units.  Ficure  ?  is  t.oe  display 
at  ^are  minute  4b  and  is  a  compendium  of  all  reports 
receiver  to  that  point.  Close  observation  of  the  screen  as 
the  s'raphics  were  heir.*;  drawn  shewed  a  sequential  enteric;; 
cf  the  information  as  it  was  being  drawn.  Sere  reasure  of 
order  was  discernible  if  t:.is  process  was  carefully 
followed,  now  ever  the  display  is  ur r ea  da  ble  after  it  is 
completed  . 

A  modification  to  the  display  pacica^e  is  required.  Cr.e 
suggestion  fror  the  test  subjects  was  that  only  the  last  two 
occurrences  of  a  contact  be  displayed,  with  earlier  reports 
available  for  historical  review  as  directed  by  function  Y. ey 
selection. 

A  benefit  of  STAMMI?2  was  demonstrated  during  the 
experiment.  In  every  iteration  of  the  display,  intelli.per.ee 
information  in  the  form  of  contact  reports  received  frer 
external  sources  was  present  cn  the  screen  as  a  reminder 
tc  the  decision  raiter.  Eecause  a  suspension  ocr.cernir.-r 
t.nis  information  is  created  in  memory,  this  information  was 
available  at  all  ti^es  for  review  an"  explanation  as 
desired.  This  information  was  not  subject  to  update  in  this 
scenario  and  did  net  seem  to  contribute  to  toe  .general 
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clutter  discussed  above. 
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There  appeared  to  be  a  disparity  between  the  STAMPER? 
conclusions  and  tncse  ci'  the  test  subjects  (questions  t  ar.d 
IT;.  The  difficulty  was  .rest  apparent  in  cases  where  the 
test  subjects  were  able  to  cross-fix  passive  detection 
tearing  lines  tc  a  geographic  pcsiticr..  Rather  than  conclude 
that  an  emitting  platform  exists  at  that  point  of 
intersection,  STAGERS  would  continue  tc  Take  conclusions 
based  only  on  the  individual  ?W  reports.  Here  the  rajor 
difficulty  appeared  tc  be  in  the  construction  of  STANMR2 
rules  rather  than  the  functioning  of  the  rules  that  exist. 
A  rule  correlating  or  pcsitlon-f  ixirg  multiple  passive 
detections  of  the  same  emitter  woula  appear  to  be  a 
relatively  simple  matter. 

The  explanation  trace  feature  cf  STa^KFES  proved  to  be 
valuable  as  both  a  decision  aid  and  as  a  trailing  tool. 
While  the  users  commented  (question  9)  that  the  trace 
included  extraneous  information,  the  level  cf  understanding 
cf  the  scenario  increased  markedly  due  to  the  increased 
appreciation  cf  what  rules  were  bein^  applied  and  hew 
STALER!  applied  them. 

The  low  average  score  for  question  11  does  not  fit 
the  general  pattern  cf  t.»e  cues t icnr.ai re  due  to  the 
wording  of  the  question  .  The  negative  response  indicate* 
that  STAKMR2  was  not  distracting  to  the  user.  The  potential 
exists  for  the  test  subjects  to  became  engrossed  in  the 
execution  cf  S?Ay.v-R2  and  the  explanation  trace  tc  the  point 


where  the  conduct  cf  the  ga^e  becomes  a  miner  consideration. 
This  apparently  was  r.ot  the  case  here. 

The  remainder  cf  the  averafees  appear  to  he  generally 
consistent  with  favorable  response  to  both  the  scenario  in 
general  and  to  STA!'/k!ZR2  in  particular. 

This  experiment  depends  heavily  on  subjective 
evaluation.  While  the  scenario  and  operation  of  STAMPERS 
appear  to  have  been  convincing  to  the  test  subjects,  there 
are  no  clear  measures  cf  operational  effectiveness  which  can 
he  Quantified  cr  analysed.  Given  that  the  computer  capacity 
is  available  to  the  location  of  the  decision  maker,  STAMPERS 
would  appear  to  be  a  welcome  addition  to  the  repertoire  cf 
decision  aids  available  to  a  commander.  While  this 
addresses  the  original  objective  namely  the 
demonstration  and  evaluation  of  ST.^KEP.2,  it  cannot 
full/  answer  the  question  of  overall  utility  of  the  system 
to  a  decision  maker  in  a  real  military  environment. 

Due  to  the  experimental  nature  of  STAk.kE?.?,  operator 
action  was  required  for  the  message  input  procedure. 
The  requirement  tc  leave  the  explanation  trace  before  the 
next  message  can  be  read  is  purely  artificial  and  ar. 
accepted  penalty  in  order  tc  run  the  program  simultaneously 
with  WES.  In  an  operational  system,  the  mechanism  for  the 
transfer  of  data  between  a  sensor  ana  the  decision  raker 
should  be  understood  and  automatic.  The  addition  of 
procedural  steps  such  as  function  key  or  tfrminal  inputs  in 
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crier  t  c  receive  a  sens  or  report  m  ay  t  e*-.d  tc  be 
ignored  during  tires  of  stress.  In  addition,  this  automatic 
transfer  •r’ust  be  regulated  tc  a  suitable  tire  period  between 
reports.  Reports  fror  multiple  sensors  at  short  intervals 
would  rapidly  overload  an  operator  required  tc  view  every 
report  . 

STAKMIR2,  as  an  i  nves t igat i on  into  the  utility  of 
artificial  intelligence  as  applied  to  command  and  control, 
has  progressed  beyond  the  concept  and  technology 
demonstration  phase  of  its  existence.  The  somewhat 
artificial  nature  of  the  message  transfer  procedures  is 
capable  of  approaching  a  "real  world'  condition  in  t.nat 
Tannine  and  human  readable  formatted  "essa^es  are  ir  ccmmer. 
use  as  in  the  P.AIKFCRM  reporting  system.  rhe 
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A  decision  aid,  even  one  directed  toward  a  unique 
user  ve.g.,  the  Tactical  Action  Officer),  ra y  need  tc  te 
distributed  througnout  a  commana  center.  The  mass  cf  data 
presented  may  require  the  decentra iizat icn  cf  control  tc  the 
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point  where  the  1AC  cr  staff  water,  officer  function  ray  te 
relegated  to  relatively  discrete  areas  of 
responsibility,  ’a h i  1  e  the  requirement  for  a  prime  decision 
maker  exists,  this  role  may  well  be  one  of  corrrani  by 
negation.  There  is  a  tire  penalty  for  reevaluating 
previously  screened  information  ir.  a  high  density 
environment.  Fach  decision  maker  would  then  require  his  own 
version  cf  STAMMB2  that  contains  a  tailored  rulp  set. 
The  present  implementation  of  STAPMF2  already  contains 
rules  which  may  be  useful  and  transportable  to  this 
practical  application. 

The  benefit  of  this  program  aboar^.  a  ship,  cr  at  a  major 
Fleet  command  center,  for  example,  could  be  that  a  "super 
agency",  consisting  cf  intelligence  and  warfare  specialists 
could  provide  a  realistic  assessment  of  situations  which 
right  be  encountered  and  tailor  p  rot  os  i  t  ic  :.s  to  reflect 
standardized  intelligence,  doctrine,  and  tactics  ir,  a  pre- 
loaded  database  ana  system  that  would,  in  real  time,  be  an 
"oracle"  which  would  provide  assistance  and  guidance  tc  the 
decision  maker.  Ir.  case  of  sue  st  ier.eble  data  the  review 
capability  of  the  process  by  which  a  decision  was  reached 
would  assist  the  decision  maker  in  logically  analyzing  his 
cwr.  thought  process.  The  training  benefit  cf  the 
preposition  review  could  increase  the  competence  cf  tr.e  TAC 
who  could  see  hew  the  'panel  of  experts’  approached  the 


problem . 


Ir.  summary,  this  thesis  has  presented  bacxgrov  rid 
iafcrratior.  concerning  artificial  intelligence  ar.d 

S?Af’M?.2.  Various  testine  methods  were  presented  tc 
evaluate  the  usefulness  of  STiiPML32  and  an  experiment  was 
conducted  and  examined  which  used  cue  cf  these  methods. 
STAPMIF.2  appears  tc  be  a  useful  decision  aid  concept  with 
dreat  potential  for  further  research  and  testing. 


APPENDIX  A 

STAKKERL  GUILE 

The  following  is  a  procedural  guide  to  beginning 
tne  STAKKER2  program  for  tnis  experiment.  After  legging  into 
the  Naval  Po  stgraduate  School  Command,  Control, 

and  Communications  Laboratory  UNIX  system  ; DEC  PIP 
11/7?),  a  TELNET  link  is  established  to  the  TOPS 2?  system 
at  Naval  Ccean  Systems  Center,  San  Diego,  California,  under 
account  name  NPS2.  Following  the  prompt,  the  entries 
with  asterisks  are  entered.  Carriage  returns  (<cr>)  are 
only  required  as  indicated. 


G  stanmer2  <cr> 

_L0AD (TIM PEAK.  CCM)  * 

compiled  on  24-Feb-&l  ie:29:2 9 
PILE  CREATEL  24-Feb-fcl  12:2t:5h 
(DISFLCE  redefined) 

(IN-LANE  redefined) 

RESULT  PR  INTER  redefined, 

TZMPEAXCCMS 
(SENSCRANGZ  reset) 

<NPS2>TIKPFAK.C0K.l 

_ ( S  ETC  ECMESEIP  (SZTC  CWNSEIP  'PZLKNAP  )  )  * 

(CWNSEI?  reset) 

'.HOKES HIP  reset) 

EELKNAP 

_(5TAKMR )  * 

ftelocme  tc  version  2.P  cf  the  SPANKER  TSA  syste~. 
Kemory  file?  ^Default  is  KZVQRY.):  KZf-.JZ  <cr>  * 
fernery  initialized. 

F.ulefile?  (Default  is  RULES  .): RULES.  Ji  <rr>  * 

Pules  loaded 

what  file  would  you  lixe  tc  take  messages  frcr? 
(Default  is  SCENE. ICE;:  SCENE. JE  <^r> 

Are  you  running  on  a  Tektronix?no  * 

Dc  you  nave  a  Tektronix  available  for  display?  nc  * 


e? 


Fcssive  detection,  heard  3FS4C  at  bearing  15b. 19  Tire 
Associated  with  track  E  E  Ri  I L  E  Y 

REPORT:  3ZRKELEY  was  sighted  ir.  the  merchant  lane  L 
Cuestion?  Cuit 
Leaving  EXPLAIN 

Passive  detection.  Heard  SPS3»  at  bearing  153.19  Time 
Associated  with  track  BERKELEY 

REPORT:  BERKELEY  was  sighted  in  the  merchant  lane  L 
Cuestion?  PF.EAK 
(Explain  broken) 
tdribble] 
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DATA  BASS 

This  is  the  data  base  frcrr  which  STANMER2  will  draw 
its  first  assertions.  It  has  beer,  specifically  tailored  to 
the  experiment  scenario  ana  dees  net  reflect  real- 
world  conditions.  STAN.tSERE  will  undate  its  data  base  as 
ressaees  are  received. 


i NERCEANTLANE  LAM 21 ) 

( LOCATION  LANEl  ((53.33  -32.1) (c5. 9  -27.25)') 
(C»NS HIP  BZIKN AP ) 

:  ID  BELKNAP  FRIEND) 

( ID-ANPLIFY  BELKNAP  ML-BATTLE) 

(ID  3ERKELIT  FRIEND) 
i ID-AKPLIFY  BERKELEY  NIL-BATTLE ) 

(ID  KNOX  FRIEND ) 

(1D-AMPLI5Y  KNCX  NIL-BATTLE) 

!  ID  U001  HOSTILE) 

( ID-ANPLIFY  U'2D  1  ML-BATTLE) 

(ID  S004  FRIEND) 

;  I D-ANPLIFY  S024  ML-AUXIL, 

(ID  S005  FRIEND) 

( ID-ANPLIFY  S005  NIL-AUXIL) 

W ID  NZR1  FRIEND) 

( ID-ANPLIFY  NEF.l  NIL-AUXIL  ) 

(ID  MS31  FRIEND) 

( ID-ANPLIFY  MSR1  ML-AUXIL, 
lID  MR1  FRIEND) 

(ID-AhPLIFY  ME1  ML-AUXIL) 

MD  MER2  FRIEND) 

(ID-AFPLIFY  MSB 2  ML-AUXIL) 

(ID  ivEP.3  FRIEND) 
aD-AN.PLIFY  MEF.3  ML-AUXIL/ 

(ID  S00S  HOSTILE) 

( ID -AMPLIFY  S028  ML-BATTLE) 
l ID  S006  HOSTILE) 

( ID-ANPLIFY  S206  ML-EATTLE) 

(ID  S 007  HOSTILE) 
l  ID-AMFLIFY  S207  ML-BATTLE) 

(ID  HCST4  HOSTILE) 
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APPENDIX  C 


RULES 


The  following  are  tne  production  rules  upon  which 

STACKERS  operates.  As  lata  frcsn  tne  data  tase  cr  Tessa?es 

satisfy  the  conditions  in  each  of  the  rules  (labelled 
<I7IfO)t  the  data  stream  is  built.  * hen  sufficient  data 
exists,  the  rules  fire  and  the  appropriate  actions  are 

carried  out.  The  plain  text  PR  IN  a  CRM  statement  is  the  rran- 

readable  explanation  of  the  assertion  which  has  been  built 
in  STAMMERS. 


INHERIT 

( CONDITIONS  ((ALIAS  ♦PLAT  ♦UNKNOWN  ) 

(TYPE  ♦PLAT  *TY?) 

( ID  *FLAT  *IDl) 

( ID- AMPLIFY  ♦PLAT  *IDMP) 

(CLASS  ♦PLAT  *CLS  ) 

(MEDIUM  ♦PLAT  *MZD  )  / 

ACTIONS 

((TYPE  ♦UNKNOWN  ♦TY?) 

( ID  ♦UNKNOWN  ♦IDl ) 

( ID-AMPLI EY  ♦UNKNOWN  ♦IDMP) 

(CLASS  ♦UNKNOWN  *015) 

\ MEDIUM  ♦UNKNOWN  ♦MED  '  } 

CONE  1 .3  PRINECRM 

'If  an  unknown  Is  identified,  it  inherits  the  properties  cf 

its  identification.") 

KNOWN-PLAT 

(CONDITION'S  ( (SIGHTING  ♦?  »S  , 

(♦UNLESS*  (CWNSHIF  *F  '» ) 

( EULLY-KNOWN  *P) 

( ID-AMPLI FY  *P  *1 ; ; 

ACTIONS 

( ( IDENTIFIED  *?) ) 

CONE  1.0  PP.INEOPM 
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>ark  as  identified  if  known  witn  certainty") 

NCT-IAST-S IGFT ING-V ER2 
( CONDITIONS  ((SIGHTING  *PLAT  -Si) 
l PREDECESSOR  *S 1  *S2) 

(*NOT*  ISAMI-AS  *SZ  NIL;) 

(♦UNLESS*  (NCT-LAST  *S2))1 
ACTIONS 

( INOT-LAST  *S2) ) 

CONE  1.0  PRINFCEK 

If  a  sighting  has  a  predecessor,  tr.en  that  predecessor  is 
not  the  last  sighting."*) 

NCT-FIRST-S IGKTING 
•  CONDIT  ION'S  (^SIGHTING  *PLAT  *S  1 ) 

(♦UNLESS*  ( NCT-FIRST  *S1)) 

(PREDECESSOR  *S 1  2 ) 

(^NOT^  *  SAFE-AS  *S2  NIL)  ) ! 

ACTIONS 

( (NOT-EIRST  *S1)  ) 

CONE  1.0  PRINFORK 

"If  an  earlier  sighting  occurs  ,  .record  that  the  previous 
sighting  is  not  the  first  sighting."/ 

LAST-VIEW 

(CONDITIONS  ((SIGHTING  ♦PLAT  *S 1 } 

(♦UNLESS*  iNOT-LAST  *S1})! 

ACTIONS 

((LAST-SIGHTING  *?LAT  *S1)) 

CON  F  .99  PRINFORI^ 

"If  the  sighting  is  not  followed,  it  is  the  last  sighting. 
I  .99/  "  5 

FIRST-VIEW 

(CONDITIONS  ((SIGETING  *PLAT  *S1) 

(♦UNLESS*  INOT-FIRST  *Sl!)! 

ACTIONS 

('(FIRST-SIGHTING  *?LAT  *S1)) 

CONE  .99  FRINFORI* 

"if  the  sighting  is  not  preceaea,  it  is  the  first  sighting, 
l .99;"! 

S I vPLY-REACHAi LF 
(CONDITIONS  ((CONTACT  *C0NT) 

(FIRST-SIGHTING  *CCNT  *S1) 

(SIGHTING  ♦PLjiT  *SZ  ) 

(ID-AMPLIET  *PL AT  ML-iATTLE) 
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(*NCT*  ( S API -AS  *CCNT  *?LAT ) ) 

(♦UNLESS*  l  O' K SHI?  *PLAT;) 

(POSITION  *S1  *F1) 

(POSITION  *S 2  *PZ, 

(Tos  *si  *tii 

(TOS  *S2  *T2) 

( S!*R  *P1  *T1  *P2  *12)  ) 

ACTIONS 

( (SINPLY-WITHIN-RIACH  *S1  *52)1 
CONF  .96  PRINFORM 

"If  a  contact's  sighting  could  travel  to  a  ^ I L-BATTLE 's 
sighting,  then  they  are  simply  reachable.  (.96)") 


REACEA5LE 

(CONDITIONS  ((CONTACT  *C0NT) 

(SIGHTING  *CCNT  *S1) 

(SIGHTING  *PLAT  *S2) 

(♦NOT*  ( SANE-AS  *FLAT  *CCNTD 
(♦UNLESS*  ( CkNSHIP  *PLAT  ) ) 

( SI ^PLY-WIT KIN “REACH  *S1  *S2) 
(♦UNLESS*  (BLCCKED-FROM  *S1  *S2))) 
ACTIONS 

(  (’wITKIN-REACH  *S1  *SZ  '  I 
CONF  .97  PRINFCP.M 


If  twc  sightings  are  vithir.  reacn  cf  each  ether,  and  are 
not  blocked  by  patrol  overflights,  then  are  reachable  from 
each  other .  ( .97  ) " ) 

COULD-BF-C CM SATAN T 
(CONDITIONS  ((CONTaCT  *C0NT) 

(FIRST-SIGHTING  *C0NT  *Sl) 

( I  E-AMP  LIFT  *PLAT  ML-BATTLZ) 

(♦UNLESS*  (OWN SHIP  *PLA  T ) ) 

(LAST-SIGHTING  *?LAT  *S2) 

(WITHIN-SSACE  *S1  *S2 I } 

ACTIONS 

(  (KNOliN-CCMBATANT  *CONT)} 

CONF  .15  PRI.NfORM 

"If  a  contact's  position  could  be  reached  by  a  jenevn 
combatant,  then  the  contact  might  be  a  combatant  (.15).") 

i.OT-KNCWN-COMEATAN  T 
(CONDITIONS  ((CONTACT  *CONT) 

(♦UNLESS*  (  KNOWN-COMBATANT  *CCNT))) 

ACTIONS 

( (TYPE  *CCNT  MERCHANT)  ) 

CONF  .45  PR  I  NFOF.M 


72 


If  a  contact  could  not  be  any  known  combatant  as 
determine!;  by  rule  CCULD-E  E-CCMBATANC ) ,  then  it  may  be  a 
merchant  f .45 } .  ”  ) 

F0S5-F.PT 

(CONDITIONS  ((PATROL  *FTL ) 

(CONTACT  •CGNT) 

(SIGHTING  *CCNT  *S1) 

(SIGHTING  *?LAT  *S2  } 
llD-AhPLIFT  *PLAT  MIL-rATTLZ) 

(•UNLESS*  (CVKSHIP  ’'‘PLAT  ) ) 

(SOURCE  *S2  *?TL ) 

(*NCT*  vSAf'S-AS  *S1  *S2}} 

(•UNLESS*  (DISSIMILAR  *CCNT  •PLAT)  )  ) 

ACTIONS 

( (POSSIBLE-REPORT  *CGNT  *?TL ) ) 

CON?  .95  PR  INFORM 

If  a  patrol  s irate  a  MIL-BATTLE  platform,  and  a  contact  is 
similar  to  the  platform,  then  the  patrol  report  concerns  the 
contact .  ( .35 )  ’  ) 

BLOCKER 

(CONDITIONS  l (CCNTaCT  *CCNT) 

(SIGHTING  *CCNT  *S 1 } 

(SIGHTING  •PLAT  *S2) 

( ID-AMPLI  FY  •PLAT  ML-EATTLE) 

(•NOT*  (SAK2-AS  •CCNT  •PLAT)) 

(•UNLESS*  ( OWN SHIP  *PLAT ) ) 

(PATROL  *?TL ) 

(•UNLESS*  (POSSIBLE-REPORT  *CONT  *PTL); 
(SIGHTING  *FTL  *S3) 

( NCT-LAST  *S3 ) 

(SUCCESSOR  *S 3  *S4) 

(POSITION  *S1  *P1) 

(POSITION  *S2  *P2) 

(POSITION  *53  *?3  > 

(POSITION  *S4  *P4 / 

(ICS  *S1  *Tl ) 

( TCS  *S2  *T2 ) 

(TOS  *S3  *T3 ) 

(TOS  *S4  *T4) 

(*CR*  ( CRCSSPATHS  *P1  *P2  *P3  *P4) 

(GPAZE  *P1  *P2  *P3  *P4) ) 

(•NOT*  UINT-EFFORE  *P1  *T1  *P2  *T2  *P3  *T3  *P* 
*T4)  ) 

(•MOT*  ( *  ENT-AFTER  *P1  *T1  •FT.  *T2  *F3  *T3  *?& 

*T4) ) j 

ACTIONS 

(  (BLOCKED-FRCM  *S1  *S2 ) ) 

CON?  .3  PPINFCRM 
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'If  a  path  between,  two  sighting  has  not  been  detected  by  a 

patrol,  and  it  would  have  if  tney  were  si^htin^s  cf  tr.e  same 

vessel,  then  they  are  different  vessels.  [.9)") 

CREATEDETZCT 

COfDIT ICVS  ^ (SIGHTING  *?LAT  *SGT) 

(♦UNLESS*  (IDENTIFIED  *PLAT ) ) 

(♦UNLESS*  ( DETECT 1CN  -PLAT)) 

(SOURCE  *SGT  EW } ) 

ACTIONS 

(VE'.TS CTION  *PLAT  )  ) 

CONE  1.2  PRINFORP 

"if  the  source  of  a  sifChtins  is  EW ,  then  mark  it  detected.") 

CREATECONTACT 

(CONDITIONS  ( (SIGHTING  #PLAT  *SGT) 

(♦UNLESS*  ( IDENTIFIED  *?LAT  ) ) 

( ♦UNLESS*  v CONTACT  *FLAT)) 

(SOURCE  *SGT  RADAR)) 

ACTIONS 

( v CONTACT  *PLAT; ) 

CON?  1.2  PRINECRN* 

'If  radar  is  the  source  cf  a  platform's  slshtins,  then  the 

platform  is  a  contact.") 

CHEAT SPLAT 

> CONDITIONS  ((SIGHTING  *?LAT  *SGT) 

l*UNLISS*  (CWNSHIF  *PLAT ) ) 

( ♦UNLESS*  (FLATFCKh  ♦PLAT))) 

ACTIONS 

( (PLATE CRN  ♦PLAT)) 

CCN  F  1.2  PRINFCRU 

"Every  si^htinej  is  a  platform,  except  the  ovnship.") 

SN’ALL-CR  AFT9 

'.CONDITIONS  (  V  CONTACT  *VEO; 

(FIRST-SIGHTING  *WEC  *S1) 

(SOURCE  *S1  RADAR; 

(RANGE  *S1  *P.l} 

(LESS-THAN  *R1  8; 

(STRENGTH  *S 1  STRONG) ) 

ACTIONS 

(  (TYPE  *WH0  SUE) 

(h.ODE  *WHO  SURFACE)) 

CON  F  .  L  PP.INi  CRI* 

"if  the  ran*e  of  a  strong  radar  si^htin*  is  less  tnar.  0  nr, 
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and  it  is  the  first  sisatir.e,  then  the  contact  is  possibly  a 
su  rf  ece  sub  .  (  .  5 )  " 

c  n,  a  1 1— CRAFT  5 

"ccfriTIONS  >.  (  CON  TACT  *X  , 

(.SIGHTING  *X  *S1GHT) 

(NOT-EIRST  *S  IGHT ) 

(RANGE  *SIGHT'*R) 

{ LZSS-THAN  *R  16) 

(GREATER-THAN  *?  9) 

(STRENGTH  •SIGHT  V EAY  ) 

(SPEED  *S1GHT  *S?D) 

(•NOT*  (GREATER-THAN  *S?D  2?.),) 

ACTIONS 

( v *OR*  (TYPE  *X  FISHING) 

(TYPE  *X  PATROL) 

(TYPE  *X  SUB)  )  ) 

CON?  .16  PR  INF CRY 

If  tae  range  cf  a  weak  sighting  is  between  9  and  15,  and 
the  speed  is  less  then  22,  then  the  contact  is  possibly  c 
sub  cr  a  patrol  or  a  fishing  vessel.  (.15)) 


SPALL-CRAFTS 

(CONDITIONS  ((CONTACT  *WHC) 

(SIGHTING  **HC  *S1) 

( NOT-E  IP.ST  •  SI) 

(SOURCE  *S1  RADAR) 

(RANGE  *51  •RANGE ) 

(LSSS-TEAN  * RANGE  16) 

(GREATER-THAN  GRANGE  S) 

(STRENGTH  *S  1  WEAR) 

(SFEED  *S1  *S?ZSD ) 

(GREATER-THAN  •SPEED  22)) 

ACTIONS 

((•OR*  ■ TYFE  *>EC  SUE) 

(TYPE  *'*HC  PATROL))) 

CON?  .3  PHI  Nr  CRN 

"if  the  range  of  a  weak  radar  sighting  is  between  S  end  16, 
and  the  speed  is  greater  than  22,  then  the  contact  is 
pcssitly  a  sub  or  a  patrol.  (.3)"; 

SMAL1-CR*FT4 

(CONDITIONS  ((CONTACT  •UNKNOWN) 

(SIGHTING  *UNKNGW!.  *SIGHTING1) 

(LAND-DIST  •SIGETIKGI  *DIST) 

(SOURCE  *S I GHTI NG 1  RADAR) 

(RANGE  *SIGHTING1  •PANGS) 

(LESS-THAN  •RANGE  9) 

(GREATER-THAN  *RANGE  3) 
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(STREETS  *S  IGHTIwG  1  VIA  S) 

( LESS -THAN  *DIST  50)) 

AC  T I ON  S 

((♦OR*  (TYPE  ♦UNKNOWN  SUB) 

(TYPE  *UNKNCWN  SHORE-PATROL) 

(TYFZ  *UNKNCWN  PLEASURE ) 

(TYPE  *UNKNCWN  COMMERCIAL) 

(TYPE  *UN KNOWN  LANDING))) 

CONE  .1  PR  INBORN 

'if  the  range  cf  a  weak  sighting  is  between  3  and  9,  and  tae 
distance  from  land  of  the  sighting  is  less  than  50,  then  the 
vessel  may  he  a  sub,  a  patrol,  aMpleasure  craft,  a  landing 
craft,  cr  a  commercial  craft.  (.1)") 

SNALL—  CRAVT3 

(CONDITIONS  ((CONTACT  *UN KNOWN  ) 

(SIGHTING  *UNKNCWN  ^SIGHTING) 

(LAND-DIST  ^SIGHTING  *DIST) 

(SOURCE  ^SIGHTING  RADAR) 

(RANGE  ’SIGHTING  GRANGE) 

(LESS-THAN  GRANGE  S) 

(GREATER-THAN  GRANGE  3) 

(STRENGTH  ♦SIGHTING  WEAK) 

(GREATER-THAN  *DIST  £0)) 

ACTIONS 

((TYPE  ♦UNKNOWN  SUB)) 

CONE  .35  PRINECRf* 

If  the  range  cf  a  weas  radar  sighting  is  between  3  and  9, 
and  the  sighting  is  further_  than  £0  miles  from  land,  then 
the  contact  is  a  sub.  (.3£)") 

S^ALL-C RA  Fm2 

(CONDITIONS*  ((CONTACT  ♦UNKNOWN) 

(SIGHTING  ♦UNKNOWN  *S IGHTING ) 

( NOT-F IRST  ♦SIGHTING) 

(SOURCE  ♦SIGHTING  RaDAR) 

(RANGE  ♦SIGHTING  ♦RANGE) 

( LESS-TEAN  ♦RANGE  3) 

(STRENGTH  ♦SIGHTING  WEAK) 

(SPEED  ♦SIGHTING  ♦SPEED) 

( ♦NOT*  (GREATER-THAN  *SPEED  3,.; 

ACTIONS 

((♦OR*  (TYPE  ’UNKNOWN  DEBRIS) 

(TYPE  ♦UNKNOWN  SUB) 

(TYFE  ’UNKNOWN  BUOY))) 

COME  .12  PRINFORN 

"if  the  weak  radar  sighting  is  not  known  to  be  moving  faster 
than  3  knots,  then  the  contact  is  either  a  buoy,  a  sub,  or 
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debris  .  (  .12  ; " ) 

5.‘- AIL-CRAiTl 

(CONDITIONS  ((CCNTaCT  *UNKNC*N) 

(SIGHTING  *U NiNCW N  ^SIGHTING} 

(NGT-HRST  ^SIGHTING) 

(  SOURCE  ^SIGHTING  RaEA?.  ) 

(RANGE  •SIGHTING  ^RANC-S) 

(LESS -THAN  GRANGE  2) 

(STRENGTH  ^SIGHTING  WEAK  5 
(SPIED  •SIGHTING  •SPEED} 

(GREATFR-THaN  •SPIED  2)) 

ACTIONS 

((TYPE  *UN KNOWN  SUB } 

(MODE  •UNKNOWN  PERISCOPES  C?.%  SNORKEL)) 

CCNF  .6  PRINFCEM 

"I?  the  weak  radar  si.=;htin-:  is  moving  at  greater  than  2 
knots,  then  tne  contact  is  a  sub  in  either  periscope  or 
s  r.crkel  mode  .  (.€}’} 

ID-LANE 

(CONDITIONS  ((SIGHTING  *SHI?  ^SIGHTING) 

(MERCHAN'TLANE  *LANE ) 

(PLATFORM  *SEIP) 

( ICC  ATI  CN  •LANE  *L  ANILOC) 

(POSITION  ^SIGHTING  *PCS ■ 

(IN-LANE  *LANIL0C  •PCS ) ) 

ACTIONS 

( ( INS  I DZ-A-MERCKANTLANS  •SIGHTING, 

(•REPORT*  *SHI?  ”  w as  sighted  ir.  the  merchant  lane 
CCNF  l.e  PR1NFCPM 

"if  a  ship  is  sighted  within  some  me rchen tlane ,  then  record 
that  it  is  inside  that  lane."; 

1 5  T  T  7  - A-S TORM 
(CONDITIONS  ( (STORM  *STCRM  ) 

(PLATFORM,  •SHI?) 

( ♦UNLESS*  i I  DENT  1  x  I ZI  •SHIP}} 

(LOCATION  •STORM  *STMLCC) 

(SIGHTING  •SHI?  •SIGHTING) 

(POSITION  *S  IGETI.NG  •PCS  ; 

( INSIDE  *?CS  *STMLCC)  i 
ACTIONS 

( \ ty?iv*seip  MERCHANT) 

(•REPORT*  •SHIP  ”  was  sighted  inside  "  *STCRM)> 
CCNF  -.25  PR  INFORM 

’If  a  ship  is  sighted  inside  a  storm,  then  the  ship  rav  r.ot 
be  a  merchant.  (.25)’) 
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•LAN 


close- popup 

(CONDITIONS  ( ( CONTACT  *SLIP) 

(FIRST-SIGHTING  *5 HIP  •SIGHTING) 
l RANGE  ‘SIGHTING  ♦RANGE) 

(LESS-TBAN  ‘RANGE  12)) 

ACTIONS 

( vTYPe"*SHI?  NSECEANT), 

CONE  -.2  PRINFGRK 

‘if  the  first  sighting  cf  a  ship  is  within  12  nn,  then  it 
ray  not  be  a  merchant.  (.2)") 

DISTANT-POPUP 

i  CONDITIONS  ( (CONTACT  *SfcIP) 

( FIRST-SIGHTING  ‘SHIP  ♦SIGHTING) 

(RANGE  ♦SIGHTING  ♦RANGE) 

IGREATER-TFAN  ♦RANGE  20}) 

ACTIONS 

((TYPE  ♦SHIP  MERCHANT)) 

CONE  -  .2  PEINE CR^' 

"if  the  range  of  the  first  sighting  is  greater  than  30  nr, 
then  the  ship  might  not  be  a  merchant.  (.2)") 

COURSE-CHANGED 
(CONDITIONS  ((CONTACT  ♦SHIP; 

(SIGHTING  ♦SHIP  ♦SIGETINGl ) 

(N'OT-FIRST  ♦SIGETINGl) 

(N0T-1AST  ♦SIGETINGl) 

(SUCCESSOR  ^SIGETINGl  ♦SIGFTING2; 

(COURSE  ♦SIGHTING1  ♦CCURSE1) 

(COURSE  ♦SIGHTING2  ♦C0URSZ2) 

( ♦UNLESS*  ( ROUGHLY -TEE -SANE- COURSE- AS  ♦COCRS21 

♦COURSES ) ) ) 

ACTIONS 

(  (TYPE  * S E I P  MRCEANT}  ) 

CONE  -.3  PRINFCP.N 

If  the  course  has  changed,,*  ignif icantly ,  then  the  sighting 
may  not  be  a  merchant.  (.3)") 

SpT£- —Q H ANGFD 

. CONDITIONS ’(( CONTACT  *SEIP) 

(SIGHTING  ♦SHIP  ♦SIGHTING) 

(NCT-HRST  ♦SIGHTING) 

(NOT-LAST  ♦SIGHTING) 

(SUCCESSOR  ♦SIGHTING  ♦SIGHTING2) 

(SPEED  ♦SIGHTING  ♦SPEZDl) 

(SPEED  ♦SIGETING2  ♦SPESD2 } 

(♦UNLESS*  ( RCUGELY-T  H  E-SAM- S  PEED- AS  *S?EED1 
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’SPEZD2)  )  ) 

ACTIONS 

(  ( TYPE  ’SHIP  MERCHANT, 

CONE  -.3  PRINFCRM 

If  the  speed  has  changed, significantl y,  then  the  sighting 
may  not  he  c  merchant.  (.3)”) 

£  A  S  T  E  R  -  T  H  A  N -  k  -  M  E  ?.  C  H  A  N  T 
(CONDITIONS  ((CONTACT  ’SHIP) 

(SIGHTING  ’SHIP  ’SIGHTING) 

(NCT-fIRST  ’SIGHTING  } 

(SPEED  ’SIGHTING  ’SPEED  ) 

(GRIaTIH-THAN  ’SPEED  2d)) 

ACTIONS 

((TYPE  ’ S K I F  MERCHANT)) 

CONE  -.2d  PPINFCRM 

’if  the  speed,,  is  greater  than  25  Ancts,  then  it  is  net  a 
merchant.  (.25)") 

slcwer-than-a-mercsant 

.CONDITIONS  ( v CONTACT  ’SLIP) 

(SIGHTING  ’SHI F  ’SIGHTING) 

(NOT-PIRST  ’SIGHTING) 

(SPEED  ’SIGHTING  ’SPEED 
(LESS-THAN  ’SPEED  9)) 

ACTIONS 

((TYPE  ’SHIP  MERCHANT)) 

CONE  -.Id  PRINFCRM 

’if  the  speed,,  is  less  than  9  Ancts,  then  it  ^ay  net  he  a 
merchant .  ( . Id  )  "  ) 

MATCH -PLAT 

CONDITIONS  ( (FIRST-SIGHTING  ’PLATA  ’SGTl) 

(NCT-LAST  ’SGTl) 

(SUCCESSOR  ’SGTl  ’SGT1S  ) 

(LAST-SIGHTING  ’FLAT2  ’SC-T2 ) 

(’UNLESS’  ( OWN SHI?  ’PLAI1)) 

(’UNLESS’  (OWNS KIP  ’PLAT2  )  ) 

(’NOT’  (SAME-AS  ’PLAT1  ’PLAT2 ) ) 

(FOSITION  ’SGT1S  ’PCS1S) 

(SPEED  ’SGT1S  ’SPD1) 

(POSITION  ’SGTl  ’P0S1  ) 

(COURSEERCM  ’PCS1  ’PCS1S  ’CESl) 

(TOS  ’SGTl  ’Tl) 

(POSITION  ’SGT2  ’P0S2  ; 

(TOS  ’SGT2  ’T2) 

(LESS-THAN  *T2  ’Tl) 

(COURSSFRCM  ’P0S2  ’POS1  ’CRS2) 
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( SPIIDPRCN  *PCS2  #T2  *PCS1  *T  1  *SPD2) 

(  RCUGHIY-THZ-S  AN'E-CGURS  I-AS  *C?.S1  *CSS2) 
(EOUGELT-TES-SAME-SPZZE-AS  *SPD1  *SFD2)) 

ACTIONS 

((ALIAS  *?LAT2  *PIAT1)) 

CON?  . t  PRINiORN. 

"if  t ne  course  and  speed  of  two  sightings  are  roughly  the 
same,  and  if  cne  sighting's  position  would  be  the  ether 
sighting's  extrapolated  position,  then  the  two  sightings  ere 
of  the  sane  vessel.  (.5)  ) 

CUTS  1 DI-ALL-LAN RS 

(CONDITIONS  ( (SIGHTING  *SHIF  ^SIGHTING) 

(*UNLSSS*  (irSNTIflZD  -SHI?)) 
iPLATiOHf  *SH I F  ) 

(♦UNLESS*  (MEDIUM  *SHIP  AIR)) 

(  *UNLISS*  (INSIDI-A-MRCHANTLANI  *S  IGETING  ) )  ) 
ACTIONS 

(  (TYPE  *SHIP  MRCEANT  )  ) 

CON  F  -.08  PRINFORh 

"If  e  sighting  is  outside  all  merchant  lanes,  then  the 
vessel  might  not  be  a  merchant.  (.08)") 

STOP 
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APPENDIX 


N'ZSS.tiOZS 


^his 

is  the  ressa^e 

file 

which  STAF”'YZR2 

will 

read,  in 

sequential  order 

(vice 

tine  o  rde  r ) ,  during 

the 

execution 

of  the  scenario. 

It  was 

built  b/  putting 

>13 

generated  reports  into  STAMPERS  readable  forr. 


(BELKNAP  64.316  -28.97  2) 

( BERKELEY  RADAR  64.2  -29.266  02) 

(KNOX  RADAR  64.3  -26.615  22) 

( ivIRl  RADAR  64.2  -20.2  2  ) 

(KERR  RADAR  64.822  -29.167  2) 

(1221  IV  270.2  K57  24) 

( Z221  EXTERNAL  274.0  BERKELEY  54.2  -29.10  05) 

( E222  EW  92.2  K2  12) 

(E222  EXTERNAL  92.0  BERKELEY  64.250  -29.15  14) 
(1001  EXTERNAL  267.0  BERKELEY  64.232  -29.122  13) 
(BELKNAP  64.4  -26.97  22) 

(KNOX  RADAR  64.267  -26.754  23) 

(EZei  EXTERNAL  263.2  BERKELEY  54.267  -29.157  20' 
(£222  EXTERNAL  92.2  BERKELEY  64.267  -29.157  20) 
(S224  EXTERNAL  64  71c  -29.222  26) 

( S  204  EXTERNAL  S4.S8C  -29.425  35) 

(  U  2  0 1  EXTERNAL  64.350  -26.232  32  s 
(5ELKNA?  64.4=5  -28.97  42) 

(BERKELEY  RADAR  54.4  -29.267  42) 

(KNOX  RADAR  64.432  -26.622  40) 

(S205  ZXTEENAL  64.422  -22.25  42) 

( S  204  EXTERNAL  64.667  -29.422  40) 

'-'032  EW  542.2  DCNKA  41) 

(  ’1  EXTERNAL  226.2  BERKELEY  64.422  -29.267  42' 

1 5  c 01  EXTERNAL  64.216  -28.50  40) 

.  E003  EXTERNAL  259.2  BERKELEY  64.467  -29.20  48) 
(5225  EXTERNAL  64.467  -32.216  45) 

(3224  EXTERNAL  64.532  -29.467  45) 

(  S  0  0  5  EXTERNAL  64.465  -29.964  50) 

(S234  EXTERNAL  64.616  -29.52  52) 

( U 2 2 1  EXTERNAL  64.352  -28.646  52) 

( U  2  2 1  EXTERNAL  64.350  -26.646  50) 

( Z024  EXTERNAL  47.2  KNOX  64.5  -26.552  57' 

( S  205  EXTERNAL  54.848  -29.333  59) 

■ E004  EXTERNAL  47.0  KNOX  64.5  -26.534  59) 
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'SELENA*  54. £70  -25.97  60) 

(BERKELEY  RADAR  64.5  -29.357  60) 

UNOX  RADAR  64.5  -25.534  50) 

f«002  EXTERNAL  005.0  2IRKELEY  54.524  -29.252  65 
( S  0  0  5  EXTERNAL  54 . 524  -25.52?  65) 

( S 004  EXTERNAL  54.570  -29.60  55} 

(S006  EXTERNAL  54.522  -29.232  65} 

(E234  EXTERNAL  4c. 0  KNCX  64.551  -25.455  70' 

( S003  EXTERNAL  000.0  BERKELEY  64.570  -25.422  75 

(U3ei  EXTERNAL  64.267  -25.750  75} 

(500?  EXTERNAL  64.76c  -27.566  74) 

(Sees  EXTERNAL  64.570  -29.616  75) 

(5005  EXTERNAL  64.267  -29.549  75} 

(BELKNAP  64.622  -25.97  5 0) 

( BERKELEY  RADAR  64.57  -29.222  82; 

(KNOX  RADAR  64.586  -28.422  60) 
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APPENDIX  i 
SCENARIO 


This  is  me  scenario  vhic 

h  will  he 

icenerat 

ec.  by  the 

1  a  r  f 

are  Er  v  ironner. 

tal  Simulator  (WES/  p 

re-ran , 

resident  in 

t.ne 

TIMX  system  at 

NCSC,  San 

Die^c,  California. 

This  file 

sets 

initial  ship 

identity, 

pcs  it icn , 

course 

ar.l  speed. 

and 

a  basic  file  of 

orders  to 

he  carried 

out  by  each  sice. 

Che  criers,  which  represent  cent  ir.^ency  plans,  are 


executed  irnediately  following  the  initialization  cf  VARGA?”' 
an  i  the  entry  cf  PLATE?,  wnich  is  the  user/interactive 
prcr:ram  for  *12. 


NORTH 

TES 

YES 

SHI? 

1.1  EIRKL  EERT 
N20001  A«AA  900 

54- 1 SN  29-05*  220  15 

Shir 

1.4.  iCNCX  KNCX 
Y.Z22Z2  EBE2  301 
64-16N  2£-50»  020  It 

SHIP 

1.2  3ZIKN  EELEN 
N00003  CCCC  SCO 

64— 1SN  2£-S5'V  020  It 

SHIP 

7.1  VOROS  TPES2 
Mllll  EEEE  904 

55- 2LU  29-20W  245  17 

SEIF 

7.2  SHORT  HASH 
N 11112  FFFF  90S 

65- 12N  29-20*  160  20 

SHIP 
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7.3  BURL  KCTLI 
N1 1 1 1 3  GGSS  926 

6c~c  5N  27 -22*  217  It 

SHIP 

7.4  VAZNY  LICFI 
Kill  14  hl'KH  92? 

64-22 K  32 -23V  142  1? 

SHIP 

7  .  c  i  o  G  N  y  C  r.  n 

Mine  mi  92c 

S-*-22N  2?-52*  2?5  25 

Sr.  IP 

7.6  YIR1  EAZBK 

N22221  922 

64-1 6N  30-16'*  222  ic 

SLIP 

?.  ?  YER2  XAZBH 

N 22222  LLLL  921 

64-52  N  29-12W  213  12 

SHI? 

?.c  MR2  EAZEK 
N22222  yt-vf  £22 
64-16N  32-22'*  226  1? 

CRLZRS 

BLUZ  PLAN  aLEA 

FCR  BZLKN  RZFCRT  ALL  ALL  Tin  1  999 
FCR  EZEKL  REPORT  ALL  nLL  Tin  1  =>99 
TCP  ENCX  F.EFCRT  ALL  ALL  TILE  1  999 
FL AC T  A  i-AR.f.Z?.  54-17N  27-45*  TIVZ  1  999 
PLACE  A  r-  ARK  HR  64-36:.  29-22*  TI  “ 

FLACE  A  BARKER  65-lSN  26-2 o*  TI 

CHANGE  FLAN  r.LZA 
EOF.  SK9RY  RZFCRT  ALL  SURFACE  TI 
FCR  VAZNY  C CURSE  24c  TIM  45 
FCR  BURL  SPZIL  *I"  r-T'  T  -c 
iCR  VCRC3  RZPCR 
FCR  BURL  RZPCRT 
FCR  VAZNY  RZPCR 
FCR  SSGNa  EZFCR 
FCn  v£3i  AZPCRT 
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NcVcl  Ocean  Systems  Corrana 
San  Die^c,  California  92132 
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